Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/agents-auto-label.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,12 @@ jobs:
github.event.workflow_run.repository.default_branch }}
sparse-checkout: .github/actions/agent-event-eligibility
sparse-checkout-cone-mode: false
path: eligibility-source

# Escape hatch: set mode: warning if false-negative skips appear post-merge.
- name: Check event eligibility
id: eligibility
uses: ./.github/actions/agent-event-eligibility
uses: ./eligibility-source/.github/actions/agent-event-eligibility
with:
expected-actions: opened,reopened,edited
custom-predicate: >-
Expand Down
11 changes: 8 additions & 3 deletions config/template-drift-allowlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@
# divergence no longer change these fingerprints, so this allowlist stops going
# stale on dependency bumps; only a genuine logic change to a root workflow will.
#
# 2026-08-05 re-baseline: auto-label now isolates the preliminary eligibility
# sparse checkout under eligibility-source/ in both root and consumer workflows.
# The remaining root/template differences are the already-reviewed consumer
# action-pin and auth-plumbing contract.
#
# 2026-07-24 re-baseline: refreshed fingerprints for 13 entries (belt 71/72/73,
# auto-label, autofix-dispatcher, capability-check, decompose, dedup, guard,
# issue-optimizer, keepalive-loop-reporter, verifier, weekly-metrics) that went
Expand Down Expand Up @@ -66,9 +71,9 @@ reason = Existing reviewed baseline drift re-baselined 2026-06-19: runtime AC me
[pair.5]
main = .github/workflows/agents-auto-label.yml
template = templates/consumer-repo/.github/workflows/agents-auto-label.yml
main_sha256 = 6276b24994010fab62c79085cc41d67ca2d0031580df2f5892c5ed2c7d1e7eba
template_sha256 = c351fecad5ee3bc4231fc33d7e0c3edbcf539f57105d649601bfbf435561b1f3
reason = Intentional divergence updated 2026-07-31: root and consumer queries now reject non-positive or invalid bounds and share executable guard tests while retaining consumer SHA pins and auth plumbing. Do not align wholesale: that would strip the consumer security contract.
main_sha256 = 2b69ddf783e49273b966ab87f0caaf65c0a3c2b803bf46eba240d0637eeda00b
template_sha256 = 60a5f4bff00adcf4ab9fe967f6f0d4b53d8be8d1111c545aa0778c7424ff69b9
reason = Intentional divergence updated 2026-08-05: root and consumer workflows both isolate the eligibility sparse checkout under eligibility-source/ while retaining consumer SHA pins and auth plumbing. Do not align wholesale: that would strip the consumer security contract.

[pair.6]
main = .github/workflows/agents-autofix-dispatcher.yml
Expand Down
10 changes: 10 additions & 0 deletions docs/INTEGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -708,6 +708,16 @@ python scripts/workflow_startup_failure_diagnostic.py --repo OWNER/REPO --run-id
This inspects check-runs for the same head SHA/run ID and prints the parser error
title/summary text that is not visible in `actions/runs/<id>/jobs`.

The same diagnostic also recognizes `action_required` runs with zero jobs.
When the run event is a public-fork `pull_request` and `head_repository.fork`
is true, classify it as a fork contributor approval hold (REST
`/actions/runs/{id}/approve` can recover it). Otherwise treat it as GitHub's
unproven-workflow protection: review the workflow file and use **Approve and
run** from an authenticated GitHub web session; the fork-PR REST approval
endpoint does not cover that class of hold. If the event is `pull_request` but
fork status cannot be determined, report an unspecified approval hold and
inspect `event` + `head_repository` before choosing remediation.


### Startup Failure (Caller Workflow Permissions)

Expand Down
15 changes: 15 additions & 0 deletions docs/ops/CONSUMER_REPO_MAINTENANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,21 @@ For bugs affecting multiple repos, create a tracking issue with:
**Example**: `workflow_run` trigger without handler job
**Detection**: Search for trigger in `on:` block, verify matching job exists

### Sparse checkout state leaking into a later checkout

**Pattern**: A job checks out one local action with `sparse-checkout`, then runs a
second `actions/checkout` into the same workspace and expects the complete
repository to be present.

**Problem**: The later checkout can retain the first checkout's sparse worktree
configuration. A subsequent local action then fails before useful work begins
with `Can't find 'action.yml'`, even though the action is present on the
repository's default branch.

**Fix**: Put the preliminary sparse checkout in a dedicated `path:` and invoke
the local action from that path. Reserve the workspace root for the later full
checkout. The auto-label workflows use `eligibility-source/` for this reason.

### Hardcoded Values

**Pattern**: Repository-specific values in templates
Expand Down
71 changes: 69 additions & 2 deletions scripts/workflow_startup_failure_diagnostic.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Capture startup_failure details for a workflow run via GitHub check-runs."""
"""Diagnose zero-job workflow startup failures and approval holds."""

from __future__ import annotations

Expand Down Expand Up @@ -98,6 +98,9 @@ def diagnose_startup_failure(repo: str, run_id: int) -> dict[str, Any]:

jobs = jobs_payload.get("jobs", [])
jobs_count = len(jobs) if isinstance(jobs, list) else 0
approval_hold = None
if run_payload.get("conclusion") == "action_required" and jobs_count == 0:
approval_hold = _classify_zero_job_approval_hold(repo, run_id, run_payload)
return {
"repo": repo,
"run_id": run_id,
Expand All @@ -106,10 +109,69 @@ def diagnose_startup_failure(repo: str, run_id: int) -> dict[str, Any]:
"run_status": run_payload.get("status", ""),
"head_sha": head_sha,
"jobs_count": jobs_count,
"approval_hold": approval_hold,
"startup_failures": findings,
}


def _head_repository_is_fork(repo: str, run_payload: dict[str, Any]) -> bool | None:
"""Return True/False when fork status is known; None when it cannot be told."""
head_repo = run_payload.get("head_repository")
if not isinstance(head_repo, dict):
return None
if "fork" in head_repo:
return bool(head_repo.get("fork"))
head_full = str(head_repo.get("full_name") or "").strip()
if not head_full:
return None
return head_full.lower() != repo.lower()


def _classify_zero_job_approval_hold(
repo: str, run_id: int, run_payload: dict[str, Any]
) -> dict[str, str]:
"""Distinguish fork-PR REST-approvable holds from unproven-workflow web holds."""
approval_url = f"https://github.com/{repo}/actions/runs/{run_id}"
event = str(run_payload.get("event") or "").strip()
is_fork = _head_repository_is_fork(repo, run_payload)

if event == "pull_request" and is_fork is True:
return {
"failure_phase": "pre_job_workflow_approval",
"suspected_root_cause": "fork_contributor_approval_hold",
"approval_url": approval_url,
"remediation": (
"Public-fork pull-request runs awaiting contributor approval can be "
"recovered with POST /repos/{owner}/{repo}/actions/runs/{run_id}/approve "
"(or Approve and run in the GitHub UI)."
),
}

if event == "pull_request" and is_fork is None:
return {
"failure_phase": "pre_job_workflow_approval",
"suspected_root_cause": "workflow_approval_hold_unspecified",
"approval_url": approval_url,
"remediation": (
"Zero-job action_required on a pull_request run: inspect event and "
"head_repository.fork before choosing remediation. Fork contributor "
"holds accept the workflow-run approval REST endpoint; unproven-workflow "
"holds require Approve and run in an authenticated GitHub web session."
),
}

return {
"failure_phase": "pre_job_workflow_approval",
"suspected_root_cause": "github_unproven_workflow_protection",
"approval_url": approval_url,
"remediation": (
"Review the workflow file, then use Approve and run from an "
"authenticated GitHub web session. The workflow-run approval "
"REST endpoint does not cover this protection."
),
}


def _classify_startup_failure(summary: str, title: str, text: str) -> tuple[str, str]:
"""Best-effort classification for parse-time startup failures."""
blob = "\n".join((title, summary, text)).lower()
Expand Down Expand Up @@ -148,11 +210,16 @@ def main(argv: list[str] | None = None) -> int:
return 1

print(json.dumps(report, indent=2))
if report["approval_hold"]:
return 0
if report["jobs_count"] == 0 and report["startup_failures"]:
return 0
if report["startup_failures"]:
return 0
print("No matching startup_failure check-runs found for this run.", file=sys.stderr)
print(
"No matching startup_failure check-runs or zero-job approval hold found " "for this run.",
file=sys.stderr,
)
return 2


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,12 @@ jobs:
github.event.workflow_run.repository.default_branch }}
sparse-checkout: .github/actions/agent-event-eligibility
sparse-checkout-cone-mode: false
path: eligibility-source

# Escape hatch: set mode: warning if false-negative skips appear post-merge.
- name: Check event eligibility
id: eligibility
uses: ./.github/actions/agent-event-eligibility
uses: ./eligibility-source/.github/actions/agent-event-eligibility
with:
expected-actions: opened,reopened,edited
custom-predicate: >-
Expand Down
116 changes: 115 additions & 1 deletion tests/scripts/test_workflow_startup_failure_diagnostic.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ def fake_gh_api(path: str, token: str | None = None) -> dict[str, Any]:
report = diag.diagnose_startup_failure("owner/repo", 555)

assert report["jobs_count"] == 0
assert report["approval_hold"] is None
assert report["head_sha"] == "abc123"
assert len(report["startup_failures"]) == 1
finding = report["startup_failures"][0]
Expand Down Expand Up @@ -199,7 +200,120 @@ def fake_gh_api(path: str, token: str | None = None) -> dict[str, Any]:
exit_code = diag.main(["--repo", "owner/repo", "--run-id", "111"])

assert exit_code == 2
assert "No matching startup_failure check-runs found" in capsys.readouterr().err
assert "No matching startup_failure check-runs or zero-job approval hold" in (
capsys.readouterr().err
)


def test_diagnose_zero_job_action_required_as_web_approval_hold(monkeypatch) -> None:
responses = [
{
"head_sha": "abc123",
"name": "Auto-Label Issues",
"conclusion": "action_required",
"status": "completed",
"event": "issues",
},
{"jobs": []},
{"check_runs": []},
]

def fake_gh_api(path: str, token: str | None = None) -> dict[str, Any]:
return responses.pop(0)

monkeypatch.setattr(diag, "_gh_api", fake_gh_api)
report = diag.diagnose_startup_failure("owner/repo", 555)

assert report["approval_hold"] == {
"failure_phase": "pre_job_workflow_approval",
"suspected_root_cause": "github_unproven_workflow_protection",
"approval_url": "https://github.com/owner/repo/actions/runs/555",
"remediation": (
"Review the workflow file, then use Approve and run from an "
"authenticated GitHub web session. The workflow-run approval "
"REST endpoint does not cover this protection."
),
}


def test_diagnose_zero_job_fork_pr_as_rest_approvable_hold(monkeypatch) -> None:
responses = [
{
"head_sha": "abc123",
"name": "CI",
"conclusion": "action_required",
"status": "completed",
"event": "pull_request",
"head_repository": {"full_name": "contributor/repo", "fork": True},
},
{"jobs": []},
{"check_runs": []},
]

def fake_gh_api(path: str, token: str | None = None) -> dict[str, Any]:
return responses.pop(0)

monkeypatch.setattr(diag, "_gh_api", fake_gh_api)
report = diag.diagnose_startup_failure("owner/repo", 555)

assert report["approval_hold"] == {
"failure_phase": "pre_job_workflow_approval",
"suspected_root_cause": "fork_contributor_approval_hold",
"approval_url": "https://github.com/owner/repo/actions/runs/555",
"remediation": (
"Public-fork pull-request runs awaiting contributor approval can be "
"recovered with POST /repos/{owner}/{repo}/actions/runs/{run_id}/approve "
"(or Approve and run in the GitHub UI)."
),
}


def test_diagnose_zero_job_pr_without_head_repo_is_unspecified(monkeypatch) -> None:
responses = [
{
"head_sha": "abc123",
"name": "CI",
"conclusion": "action_required",
"status": "completed",
"event": "pull_request",
},
{"jobs": []},
{"check_runs": []},
]

def fake_gh_api(path: str, token: str | None = None) -> dict[str, Any]:
return responses.pop(0)

monkeypatch.setattr(diag, "_gh_api", fake_gh_api)
report = diag.diagnose_startup_failure("owner/repo", 555)

hold = report["approval_hold"]
assert hold is not None
assert hold["suspected_root_cause"] == "workflow_approval_hold_unspecified"
assert "head_repository.fork" in hold["remediation"]


def test_main_accepts_zero_job_action_required_hold(monkeypatch, capsys) -> None:
responses = [
{
"head_sha": "abc123",
"name": "Auto-Label Issues",
"conclusion": "action_required",
"status": "completed",
"event": "issues",
},
{"jobs": []},
{"check_runs": []},
]

def fake_gh_api(path: str, token: str | None = None) -> dict[str, Any]:
return responses.pop(0)

monkeypatch.setattr(diag, "_gh_api", fake_gh_api)
exit_code = diag.main(["--repo", "owner/repo", "--run-id", "555"])

assert exit_code == 0
assert "pre_job_workflow_approval" in capsys.readouterr().out


def test_main_returns_1_when_diagnosis_raises(monkeypatch, capsys) -> None:
Expand Down
36 changes: 36 additions & 0 deletions tests/workflows/test_auto_label_checkout_isolation.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
from pathlib import Path

import yaml

WORKFLOWS = (
Path(".github/workflows/agents-auto-label.yml"),
Path("templates/consumer-repo/.github/workflows/agents-auto-label.yml"),
)


def _steps(path: Path) -> list[dict[str, object]]:
workflow = yaml.safe_load(path.read_text(encoding="utf-8"))
return workflow["jobs"]["auto-label"]["steps"]


def test_auto_label_isolates_eligibility_sparse_checkout() -> None:
for path in WORKFLOWS:
steps = _steps(path)
eligibility_checkout = next(
step for step in steps if step.get("name") == "Checkout eligibility action"
)
eligibility = next(step for step in steps if step.get("name") == "Check event eligibility")

assert eligibility_checkout["with"]["path"] == "eligibility-source", path
assert (
eligibility["uses"] == "./eligibility-source/.github/actions/agent-event-eligibility"
), path


def test_auto_label_full_checkout_keeps_workspace_root() -> None:
for path in WORKFLOWS:
steps = _steps(path)
full_checkout = next(step for step in steps if step.get("name") == "Checkout repository")

assert "path" not in full_checkout.get("with", {}), path
assert any(step.get("uses") == "./.github/actions/setup-api-client" for step in steps), path
Loading