Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 11 additions & 4 deletions .github/scripts/issue_format.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@
sit inside a ```bash fence validates as conforming — a false negative that lets
an unactionable issue through the guard. Well-written issues quote commands and
expected output in fences constantly, so this is the common case, not an edge
one. Any change to heading detection must keep a fenced-heading case in
tests/scripts/test_issue_format.py.
one. Any change to heading detection must keep a fenced-heading case in the
upstream Workflows test `tests/scripts/test_issue_format.py`.

Pure stdlib on purpose — it must run on a bare runner with no install step.
"""
Expand Down Expand Up @@ -83,7 +83,14 @@ def _headings(body: str) -> list[tuple[str, int, int]]:
marker = fence_match.group(1)
if fence is None:
fence = (marker[0], len(marker))
elif marker[0] == fence[0] and len(marker) >= fence[1]:
elif (
marker[0] == fence[0]
and len(marker) >= fence[1]
and re.fullmatch(
rf"\s{{0,3}}(?:`{{{fence[1]},}}|~{{{fence[1]},}})\s*",
line,
)
):
fence = None
continue
if fence is not None:
Expand All @@ -96,7 +103,7 @@ def _headings(body: str) -> list[tuple[str, int, int]]:

def _find(body: str, aliases: tuple[str, ...]) -> int | None:
for text, idx, _ in _headings(body):
if text in aliases:
if any(text == alias or text.startswith(f"{alias} (") for alias in aliases):
return idx
return None

Expand Down
2 changes: 1 addition & 1 deletion .github/sync-manifest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ workflows:
description: "Issue optimizer - LangChain-based issue formatting and optimization (Phase 1)"

- source: .github/workflows/agents-issue-format-guard.yml
description: "Issue format guard - validates issues on open/edit/reopen and hold-label changes against AGENT_ISSUE_FORMAT, while durable/wontfix and bot issues remain exempt. Pause and needs-human labels hold dispatch, clear stale agents:formatted state on held edits, and revalidate on resume. Non-conforming unheld work gets agents:format so the optimizer repairs it. Requires .github/scripts/issue_format.py."
description: "Issue format guard - validates issues on open/edit/reopen, hold/exemption-label changes, and manual dispatch against AGENT_ISSUE_FORMAT, while durable/wontfix and bot issues remain exempt. Pause and needs-human labels hold dispatch. Any invalid non-exempt issue change on those triggers clears stale agents:formatted state, and hold removal revalidates on resume. Non-conforming unheld work gets agents:format so the optimizer repairs it. Requires .github/scripts/issue_format.py."
template_sync: exact

- source: .github/workflows/agents-verify-to-new-pr.yml
Expand Down
12 changes: 7 additions & 5 deletions .github/workflows/agents-issue-format-guard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,19 +76,21 @@ jobs:
exit "$rc"
fi

- name: Invalidate stale format completion after an invalid edit
- name: Invalidate stale format completion after an invalid issue change
if: >-
steps.issue.outputs.exempt != 'true' && github.event.action == 'edited' &&
steps.validate.outputs.rc == '1'
steps.issue.outputs.exempt != 'true' && steps.validate.outputs.rc == '1'
Comment thread
stranske marked this conversation as resolved.
env:
GH_TOKEN: ${{ github.token }}
NUMBER: ${{ steps.issue.outputs.number }}
run: |
set -euo pipefail
if gh issue view "$NUMBER" --repo "$GITHUB_REPOSITORY" --json labels \
--jq '.labels[].name' | grep -qx 'agents:formatted'; then
gh issue edit "$NUMBER" --repo "$GITHUB_REPOSITORY" --remove-label "agents:formatted"
echo "Invalid issue edit — cleared agents:formatted before rerouting."
if gh issue edit "$NUMBER" --repo "$GITHUB_REPOSITORY" --remove-label "agents:formatted"; then
echo "Invalid issue body — cleared agents:formatted before rerouting."
else
echo "::warning::could not remove stale agents:formatted"
fi
fi

- name: Route non-conforming issue to the optimizer
Expand Down
2 changes: 1 addition & 1 deletion docs/ci/WORKFLOWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ Consumer default note: `agents-pr-meta-v4.yml` is a Workflows-repo service workf
* [`agents-63-issue-intake.yml`](../../.github/workflows/agents-63-issue-intake.yml) is the canonical front door. It now listens for `agent:codex` labels directly and routes both label triggers and ChatGPT sync requests through the shared normalization pipeline.
* [`agents-64-verify-agent-assignment.yml`](../../.github/workflows/agents-64-verify-agent-assignment.yml) validates that labelled issues retain an approved agent assignee and publishes the verification outputs.
* [`agents-issue-optimizer.yml`](../../.github/workflows/agents-issue-optimizer.yml) runs issue optimization passes when `agents:optimize` or `agents:apply-suggestions` labels are applied.
* [`agents-issue-format-guard.yml`](../../.github/workflows/agents-issue-format-guard.yml) validates opened, edited, reopened, and hold/exemption-label changes against `AGENT_ISSUE_FORMAT`; durable/wontfix and bot issues remain exempt. The validator requires the `Tasks` and `Acceptance Criteria` structure plus concrete task/verification evidence, preserves nested Markdown subsections, and reports `Why`, `Scope`, `Implementation Notes`, and `Non-Goals` as advisory. `agents:auto-pilot-pause` and `needs-human` hold optimizer dispatch, invalid edits clear a stale `agents:formatted` marker, and hold removal revalidates before restoring it. Non-conforming unheld work is explicitly dispatched to the optimizer's `format` phase.
* [`agents-issue-format-guard.yml`](../../.github/workflows/agents-issue-format-guard.yml) validates opened, edited, reopened, hold/exemption-label changes, and manual dispatch against `AGENT_ISSUE_FORMAT`; durable/wontfix and bot issues remain exempt. The validator requires the `Tasks` and `Acceptance Criteria` structure plus concrete task/verification evidence, preserves nested Markdown subsections, and reports `Why`, `Scope`, `Implementation Notes`, and `Non-Goals` as advisory. `agents:auto-pilot-pause` and `needs-human` hold optimizer dispatch. Any invalid non-exempt issue change on those triggers (opened, edited, reopened, eligible hold/exemption label events, or manual dispatch) clears a stale `agents:formatted` marker; hold removal revalidates before restoring it. Non-conforming unheld work is explicitly dispatched to the optimizer's `format` phase.
* [`agents-moderate-connector.yml`](../../.github/workflows/agents-moderate-connector.yml) moderates connector-authored PR comments, enforcing repository allow/deny lists and applying the debugging label when deletions occur.
* [`agents-guard.yml`](../../.github/workflows/agents-guard.yml) applies repository-level guardrails before agent workflows run.
* [`pr-46-dependency-repair-contract.yml`](../../.github/workflows/pr-46-dependency-repair-contract.yml) keeps clean Renovate/Dependabot branches bot-owned and validates the bot-delta first commit on agent-owned dependency repair promotions. See [`docs/ops/DEPENDENCY_REPAIR_PROMOTION.md`](../ops/DEPENDENCY_REPAIR_PROMOTION.md) for the lane-selection contract and marker.
Expand Down
15 changes: 11 additions & 4 deletions templates/consumer-repo/.github/scripts/issue_format.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@
sit inside a ```bash fence validates as conforming — a false negative that lets
an unactionable issue through the guard. Well-written issues quote commands and
expected output in fences constantly, so this is the common case, not an edge
one. Any change to heading detection must keep a fenced-heading case in
tests/scripts/test_issue_format.py.
one. Any change to heading detection must keep a fenced-heading case in the
upstream Workflows test `tests/scripts/test_issue_format.py`.

Pure stdlib on purpose — it must run on a bare runner with no install step.
"""
Expand Down Expand Up @@ -83,7 +83,14 @@ def _headings(body: str) -> list[tuple[str, int, int]]:
marker = fence_match.group(1)
if fence is None:
fence = (marker[0], len(marker))
elif marker[0] == fence[0] and len(marker) >= fence[1]:
elif (
marker[0] == fence[0]
and len(marker) >= fence[1]
and re.fullmatch(
rf"\s{{0,3}}(?:`{{{fence[1]},}}|~{{{fence[1]},}})\s*",
line,
)
):
fence = None
continue
if fence is not None:
Expand All @@ -96,7 +103,7 @@ def _headings(body: str) -> list[tuple[str, int, int]]:

def _find(body: str, aliases: tuple[str, ...]) -> int | None:
for text, idx, _ in _headings(body):
if text in aliases:
if any(text == alias or text.startswith(f"{alias} (") for alias in aliases):
return idx
return None

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,19 +76,21 @@ jobs:
exit "$rc"
fi

- name: Invalidate stale format completion after an invalid edit
- name: Invalidate stale format completion after an invalid issue change
if: >-
steps.issue.outputs.exempt != 'true' && github.event.action == 'edited' &&
steps.validate.outputs.rc == '1'
steps.issue.outputs.exempt != 'true' && steps.validate.outputs.rc == '1'
env:
GH_TOKEN: ${{ github.token }}
NUMBER: ${{ steps.issue.outputs.number }}
run: |
set -euo pipefail
if gh issue view "$NUMBER" --repo "$GITHUB_REPOSITORY" --json labels \
--jq '.labels[].name' | grep -qx 'agents:formatted'; then
gh issue edit "$NUMBER" --repo "$GITHUB_REPOSITORY" --remove-label "agents:formatted"
echo "Invalid issue edit — cleared agents:formatted before rerouting."
if gh issue edit "$NUMBER" --repo "$GITHUB_REPOSITORY" --remove-label "agents:formatted"; then
echo "Invalid issue body — cleared agents:formatted before rerouting."
else
echo "::warning::could not remove stale agents:formatted"
fi
fi

- name: Route non-conforming issue to the optimizer
Expand Down
20 changes: 20 additions & 0 deletions tests/scripts/test_issue_format.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,26 @@ def test_fenced_headings_do_not_satisfy_required_sections() -> None:
assert set(report.missing_required) == {"Tasks", "Acceptance Criteria"}


def test_fence_with_language_marker_does_not_close_a_code_block() -> None:
validator = _validator()
report = validator.validate(
"```markdown\n## Tasks\n- [ ] pretend\n```python\n"
"## Acceptance Criteria\n- pytest tests/test_x.py\n````\n"
)
assert not report.ok
assert "Acceptance Criteria" in report.missing_required


def test_heading_with_trailing_qualifier_matches_required_section() -> None:
validator = _validator()
report = validator.validate(
VALID_CONTEXT
+ "## Tasks (in order)\n- [ ] Implement it\n\n"
+ "## Acceptance Criteria (all must hold)\n- pytest tests/test_x.py passes\n"
)
assert report.ok


def test_checkbox_and_subjective_errors_are_non_conforming() -> None:
validator = _validator()
report = validator.validate(
Expand Down
Loading