Skip to content

fix(agents): fail closed format guard revalidation #2987

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stranske merged 3 commits into from
Aug 8, 2026
Merged

fix(agents): fail closed format guard revalidation #2987

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
03f7532
fix(agents): fail closed format guard revalidation
codex-automation Aug 8, 2026
979021d
fix(agents): capture stderr on format-guard revalidation
codex-automation Aug 8, 2026
3ecb0c8
chore(autofix): formatting/lint
github-actions[bot] Aug 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 31 additions & 6 deletions .github/workflows/agents-issue-format-guard.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,13 @@ permissions:
issues: write

concurrency:
# `inputs` only exists for workflow_dispatch/workflow_call. Event inputs are
# null for issue events, so retain the manual fallback without making normal
# issue-triggered runs depend on the dispatch-only context.
group: issue-format-guard-${{ github.event.issue.number || github.event.inputs.issue_number || inputs.issue_number }}
# Event inputs are null for issue events, so retain the manual fallback
# without making normal issue-triggered runs depend on the dispatch-only
# `inputs` context.
group: >-
issue-format-guard-${{
github.event.issue.number || github.event.inputs.issue_number ||
github.run_id }}
# A skipped label event must never cancel an in-flight opened/edited check.
cancel-in-progress: false

Expand Down Expand Up @@ -50,7 +53,7 @@ jobs:
id: issue
env:
GH_TOKEN: ${{ github.token }}
NUMBER: ${{ github.event.issue.number || inputs.issue_number }}
NUMBER: ${{ github.event.issue.number || github.event.inputs.issue_number }}
run: |
set -euo pipefail
gh issue view "$NUMBER" --repo "$GITHUB_REPOSITORY" \
Expand Down Expand Up @@ -132,10 +135,32 @@ jobs:
exit 0
fi
if [[ -f .github/scripts/issue_format.py ]]; then
if python3 .github/scripts/issue_format.py body.md > report.md; then
error_file="$(mktemp)"
set +e
python3 .github/scripts/issue_format.py body.md > report.md 2> "$error_file"
revalidate_rc=$?
set -e
if [[ "$revalidate_rc" -eq 0 ]]; then
echo "Live body now conforms — skipping optimizer dispatch."
rm -f "$error_file"
exit 0
fi
# Exit 1 is used for both non-conformance and Python crashes; stderr
# distinguishes a validator runtime error (fail closed) from a normal
# non-conforming body (continue to fingerprint/optimizer).
if [[ "$revalidate_rc" -eq 1 && -s "$error_file" ]]; then
echo "::error::issue-format validator failed unexpectedly during revalidation"
cat "$error_file" >&2
rm -f "$error_file"
exit 1
fi
if [[ "$revalidate_rc" -ne 1 ]]; then
echo "::error::issue-format validator failed unexpectedly during revalidation (exit $revalidate_rc)"
cat "$error_file" >&2
rm -f "$error_file"
exit "$revalidate_rc"
Comment thread
stranske marked this conversation as resolved.
Comment thread
stranske marked this conversation as resolved.
fi
Comment thread
coderabbitai[bot] marked this conversation as resolved.
rm -f "$error_file"
fi
fingerprint="$(sha256sum body.md | cut -c1-12)"
marker="<!-- format-guard:$fingerprint -->"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/agents-issue-optimizer.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ concurrency:
group: >-
agents-issue-optimizer-${{
github.repository }}-${{
github.event.issue.number || inputs.issue_number || github.run_id }}
github.event.issue.number || github.event.inputs.issue_number || github.run_id }}
cancel-in-progress: false

jobs:
Expand Down
4 changes: 2 additions & 2 deletions config/template-drift-allowlist.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,8 @@ reason = Intentional divergence re-baselined 2026-06-30: root and consumer guard
[pair.11]
main = .github/workflows/agents-issue-optimizer.yml
template = templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml
main_sha256 = 128b60e2af8e3957209849c7f20502a1b9a971e0c8c347e25a0d501a868bc484
template_sha256 = f7987a566c15971ae9039bb3d1a53f0bb654378e6c1dc3103f5692f5abca022d
main_sha256 = d14dacb2a76447d35295a5d8a4da1e9a56a545323f87413fd76cf0654e8cf116
template_sha256 = 05ad7bff15067f0827e136559145e3c13e0372b538889ac78281ce47658cefce
reason = Intentional divergence re-baselined 2026-08-08c: root remains in-tree (scripts/langchain + .github/scripts/issue_format.py); consumer vendors those via Workflows sparse-checkout under workflows-scripts/. Shared behavioral contract this round: concurrency includes inputs.issue_number for workflow_dispatch dedupe, cancel-safe (failure()||cancelled()) agents:format lease release, issue_dedup wrapped in set +e/set -e so advisory failures cannot abort under bash -e, and visible non-zero exit warnings. Do not align wholesale — that would strip consumer action pins/token setup.

[pair.12]
Expand Down
37 changes: 31 additions & 6 deletions templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,13 @@ permissions:
issues: write

concurrency:
# `inputs` only exists for workflow_dispatch/workflow_call. Event inputs are
# null for issue events, so retain the manual fallback without making normal
# issue-triggered runs depend on the dispatch-only context.
group: issue-format-guard-${{ github.event.issue.number || github.event.inputs.issue_number || inputs.issue_number }}
# Event inputs are null for issue events, so retain the manual fallback
# without making normal issue-triggered runs depend on the dispatch-only
# `inputs` context.
group: >-
issue-format-guard-${{
github.event.issue.number || github.event.inputs.issue_number ||
github.run_id }}
# A skipped label event must never cancel an in-flight opened/edited check.
cancel-in-progress: false

Expand Down Expand Up @@ -50,7 +53,7 @@ jobs:
id: issue
env:
GH_TOKEN: ${{ github.token }}
NUMBER: ${{ github.event.issue.number || inputs.issue_number }}
NUMBER: ${{ github.event.issue.number || github.event.inputs.issue_number }}
run: |
set -euo pipefail
gh issue view "$NUMBER" --repo "$GITHUB_REPOSITORY" \
Expand Down Expand Up @@ -132,10 +135,32 @@ jobs:
exit 0
fi
if [[ -f .github/scripts/issue_format.py ]]; then
if python3 .github/scripts/issue_format.py body.md > report.md; then
error_file="$(mktemp)"
set +e
python3 .github/scripts/issue_format.py body.md > report.md 2> "$error_file"
revalidate_rc=$?
set -e
if [[ "$revalidate_rc" -eq 0 ]]; then
echo "Live body now conforms — skipping optimizer dispatch."
rm -f "$error_file"
exit 0
fi
# Exit 1 is used for both non-conformance and Python crashes; stderr
# distinguishes a validator runtime error (fail closed) from a normal
# non-conforming body (continue to fingerprint/optimizer).
if [[ "$revalidate_rc" -eq 1 && -s "$error_file" ]]; then
echo "::error::issue-format validator failed unexpectedly during revalidation"
cat "$error_file" >&2
rm -f "$error_file"
exit 1
fi
if [[ "$revalidate_rc" -ne 1 ]]; then
echo "::error::issue-format validator failed unexpectedly during revalidation (exit $revalidate_rc)"
cat "$error_file" >&2
rm -f "$error_file"
exit "$revalidate_rc"
Comment thread
stranske marked this conversation as resolved.
fi
rm -f "$error_file"
fi
fingerprint="$(sha256sum body.md | cut -c1-12)"
marker="<!-- format-guard:$fingerprint -->"
Expand Down
2 changes: 1 addition & 1 deletion templates/consumer-repo/.github/workflows/agents-issue-optimizer.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ concurrency:
group: >-
agents-issue-optimizer-${{
github.repository }}-${{
github.event.issue.number || inputs.issue_number || github.run_id }}
github.event.issue.number || github.event.inputs.issue_number || github.run_id }}
cancel-in-progress: false

jobs:
Expand Down
30 changes: 30 additions & 0 deletions tests/workflows/test_agents_issue_optimizer_format_trigger.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,3 +102,33 @@ def test_format_lease_is_required_and_released_after_failure() -> None:
"&& steps.check.outputs.phase == 'format'"
) in text
assert 'gh issue edit "$ISSUE_NUMBER" --remove-label "agents:format"' in text


def test_format_guard_uses_event_inputs_and_fails_closed_on_revalidation_errors() -> None:
for text in (
GUARD_PATH.read_text(encoding="utf-8"),
CONSUMER_GUARD_PATH.read_text(encoding="utf-8"),
):
assert "github.event.inputs.issue_number ||\n github.run_id" in text
assert "github.event.issue.number || github.event.inputs.issue_number" in text
assert "revalidate_rc=$?" in text
assert (
'python3 .github/scripts/issue_format.py body.md > report.md 2> "$error_file"' in text
)
# Exit 0: live body now conforms — skip optimizer dispatch.
assert 'if [[ "$revalidate_rc" -eq 0 ]]; then' in text
assert "Live body now conforms — skipping optimizer dispatch." in text
# Exit 1 + stderr: validator crash — fail closed before fingerprint/dispatch.
assert 'if [[ "$revalidate_rc" -eq 1 && -s "$error_file" ]]; then' in text
assert "::error::issue-format validator failed unexpectedly during revalidation" in text
# Other non-zero: propagate; exit 1 with empty stderr continues to fingerprint.
assert 'if [[ "$revalidate_rc" -ne 1 ]]; then' in text
assert 'fingerprint="$(sha256sum body.md | cut -c1-12)"' in text

for text in (
WORKFLOW_PATH.read_text(encoding="utf-8"),
CONSUMER_WORKFLOW_PATH.read_text(encoding="utf-8"),
):
assert (
"github.event.issue.number || github.event.inputs.issue_number || github.run_id" in text
)
Loading