fix(agents): stop the format guard inverting its own label checks

[stranske]

Automated Status Summary

Scope

Address unmet acceptance criteria from PR #463.

Original scope:

• After merging PR chore(codex): bootstrap PR for issue #101 #103 (multi-agent routing infrastructure), we need to:
• 1. Validate the CLI agent pipeline works end-to-end with the new task-focused prompts
• 2. Add GITHUB_STEP_SUMMARY output so iteration results are visible in the Actions UI
• 3. Streamline the Automated Status Summary to reduce clutter when using CLI agents
• 4. Clean up comment patterns to avoid a mix of old UI-agent and new CLI-agent comments

Context for Agent

Related Issues/PRs

• stranske/Workflows#463
• #463
• #103

Tasks

• After PR chore(codex): bootstrap PR for issue #101 #103 merges, create a test PR with agent:codex label
• Verify task appendix appears in Codex prompt (check workflow logs)
• Verify Codex works on actual tasks (not random infrastructure work)
• Verify keepalive comment updates with iteration progress
• Add step summary output to agents-keepalive-loop.yml after agent run
• Include: iteration number, tasks completed, files changed, outcome
• Ensure summary is visible in workflow run UI
• Modify buildStatusBlock() in agents_pr_meta_update_body.js to accept agentType parameter
• When agentType is set (CLI agent): hide workflow table, hide head SHA/required checks
• Keep Scope/Tasks/Acceptance checkboxes for all cases
• Pass agent type from workflow to the update_body job
• For CLI agents (agent:* label):
• Suppress <!-- gate-summary: --> comment posting (use step summary instead)
• Suppress <!-- keepalive-round: N --> instruction comments (task appendix replaces this)
• Update <!-- keepalive-loop-summary --> to be the single source of truth
• Ensure state marker is embedded in the summary comment (not separate)
• For UI Codex (no agent:* label):
• Keep existing comment patterns (instruction comments, connector bot reports)
• Keep <!-- gate-summary: --> comment
• Add agent_type output to detect job so downstream workflows know the mode
• Update agents-pr-meta.yml to conditionally skip gate summary for CLI agent PRs

Acceptance criteria

• CLI agent receives explicit tasks in prompt and works on them
• CLI agent PRs have ≤3 bot comments total (summary, one per iteration update) instead of 10+
• Requires PR chore(codex): bootstrap PR for issue #101 #103 to be merged first

Summary by CodeRabbit

• Bug Fixes
  • Improved issue-format validation warnings when the validator is unavailable.
  • Made label checks more reliable during stale cleanup, held-state restoration, and format-trigger processing.
• Chores
  • Updated recorded workflow execution metadata.

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The issue-format guard workflows now warn when the validator is missing and materialize label queries before matching labels in cleanup and restoration paths. Worker attempt metadata now records an updated emission timestamp for PR 2994.

Changes

Issue format guard

Layer / File(s)	Summary
Missing validator warning .github/workflows/agents-issue-format-guard.yml, templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml	The workflows warn when the validator is absent and validation did not run.
File-backed label checks .github/workflows/agents-issue-format-guard.yml, templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml	Label query output is written to files before grep checks for stale, held, and format-trigger labels.

Worker attempt metadata

Layer / File(s)	Summary
Attempt metadata update langsmith-fleet-worker-attempt.json	The metadata updates the emission timestamp and keeps the attempt associated with PR 2994.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

• [P0] Format-guard → optimizer loop is unbounded: closed issues never exempt, run-correlation always matches zero, no attempt cap #2991: The issue addresses overlapping label-query and grep behavior, but its unbounded-loop defects are not changed here.

Possibly related PRs

• stranske/Workflows#2956: Introduces the issue-format guard workflow refined by this PR.
• stranske/Workflows#2979: Changes related issue-format guard revalidation and label-trigger cleanup.
• stranske/Workflows#2985: Changes the same workflow’s agents:format lease and recovery cleanup.

Suggested labels: verify:compare

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: correcting inverted label checks in the agents format guard.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/guard-pipefail-and-silent-skip

---

_{Comment @coderabbitai help to get the list of available commands.}

[agents-workflows-bot]

Automated Status Summary

Head SHA: d3dd85e
Latest Runs: ⏳ pending — Gate
Required contexts: summary
Required: core tests (3.12): ⏳ pending, core tests (3.13): ⏳ pending, docker smoke: ⏳ pending, gate: ⏳ pending

Workflow / Job	Result	Logs
(no jobs reported)	⏳ pending	—

Coverage Overview

• Coverage history entries: 0

Updated automatically; will refresh on subsequent CI/Docker completions.

---

Keepalive checklist

Scope

No scope information available

Tasks

• No tasks defined

Acceptance criteria

• No acceptance criteria defined

[Copilot]

Pull request overview

Fixes a correctness bug in the issue format guard workflow where gh … | grep -q pipelines could invert their condition under set -o pipefail (SIGPIPE on gh after grep -q exits early), leading to incorrect label-branch decisions and, in one case, repeated re-triggering behavior.

Changes:

• Replaces gh … | grep -qx … label checks with “materialize to file, then grep the file” to avoid pipefail/SIGPIPE inversion.
• Adds an explicit ::warning:: annotation when .github/scripts/issue_format.py is missing, while preserving the existing “skip without failing” behavior.
• Applies identical changes to both the source workflow and the consumer template copy.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
.github/workflows/agents-issue-format-guard.yml	Avoids pipefail inversion by writing gh issue view label output to files before grep, and emits a warning when the validator script is missing.
templates/consumer-repo/.github/workflows/agents-issue-format-guard.yml	Mirrors the same correctness and observability fixes in the consumer template workflow.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

Runner dispatch state for codex on PR #2994. Do not edit.

[stranske]

Closer advance (cursor)

Rebased onto current main (post-#2998) in linked worktree and force-pushed exact head 0db11d8a45f501f47d6134fa718ee3a257e629d6.

Conflict resolution kept:

• main live-body revalidation / closed-issue preserve path from fix(agents): cap issue-format optimizer retries #2998
• pipefail-safe materialised labels.txt / held.txt greps from this PR
• validator-absent ::warning signal

Validation: root/consumer YAML parse OK, byte parity OK, tests/workflows/test_issue_format_guard_closed_issues.py 7 passed.

PR body updated so this no longer claims Closes #464 (that issue is already closed). Post-push review window applies before merge.

[agents-workflows-bot]

🤖 Keepalive Loop Status

PR #2994 | Agent: Codex | Iteration 0/12

Current State

Metric	Value
Iteration progress	[----------] 0/12
Action	skip (needs-human)
Disposition	skipped
Gate	success
Tasks	0/24 complete
Timeout	45 min (default)
Timeout usage	3m elapsed (8%, 42m remaining)
Keepalive	✅ enabled
Autofix	❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | unknown |
| Suggested recovery | Capture logs and context; retry once and escalate if the issue persists. |

⚠️ Failure Tracking

| Consecutive failures | 6/3 |
| Reason | agent-run-failed |

[stranske-keepalive]

Keepalive Work Log (click to expand)

#	Time (UTC)	Agent	Action	Result	Files	Tasks	Progress	Commit	Gate
0	2026-08-09 02:45:25	Codex	run (agent-run-failed) retry	failure	2 file(s)	0	0/24	5724edb	—
0	2026-08-09 02:48:21	Codex	run (agent-run-failed) retry	failure	2 file(s)	0	0/24	3103ebd	success
0	2026-08-09 02:51:12	Codex	run (agent-run-failed-repeat) retry	failure	2 file(s)	0	0/24	711762d	success
0	2026-08-09 02:51:50	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	cancelled
0	2026-08-09 02:52:30	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	cancelled
0	2026-08-09 02:53:11	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	cancelled
0	2026-08-09 02:54:25	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	success
0	2026-08-09 02:56:26	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	success
0	2026-08-09 03:45:54	Codex	run (agent-run-failed-repeat) retry	failure	2 file(s)	0	0/24	d53a111	success
0	2026-08-09 03:46:38	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	—
0	2026-08-09 03:47:20	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	cancelled
0	2026-08-09 03:48:20	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	cancelled
0	2026-08-09 03:48:57	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	cancelled
0	2026-08-09 03:50:45	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	success
0	2026-08-09 04:30:31	Codex	run (agent-run-failed-repeat) retry	failure	2 file(s)	0	0/24	f45cdda	success
0	2026-08-09 04:31:17	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	—
0	2026-08-09 04:32:03	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	cancelled
0	2026-08-09 04:32:40	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	cancelled
0	2026-08-09 04:34:44	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	success
0	2026-08-09 04:41:12	Codex	run (agent-run-failed-repeat) retry	failure	1 file(s)	0	0/24	fa2a38a	—
0	2026-08-09 04:41:56	Codex	skip (needs-human)	skipped	—	0	0/24	—	cancelled
0	2026-08-09 04:41:57	Codex	skip (needs-human) retry	skipped	—	0	0/24	—	cancelled
0	2026-08-09 04:42:28	Codex	skip (needs-human)	skipped	—	0	0/24	—	cancelled
0	2026-08-09 04:44:40	Codex	skip (needs-human)	skipped	—	0	0/24	—	success

[stranske]

Closer recovery: rebased this branch onto current main and force-pushed exact head 3b7c5984b7b46a9efdc8f29f675e2987829be0d3. Removed stale automation-loop needs-human / agent:needs-attention labels and the false Closes #464 preamble; #464 has been closed since 2026-01-03 and is not this follow-up's source. Validation: issue-format closed-issues suite 7 passed; template completeness passed; template drift reports 0 unallowlisted; sync check passed. The mandatory seven-minute exact-head review window and fresh checks/thread reread apply before any merge.

[stranske-keepalive]

Workflow source needed

PR #2994 needs either a linked GitHub issue or one valid non-issue Workflow Source before PR metadata automation can manage it safely.

Please do one of:

• Add <!-- meta:issue:123 --> or a normal Closes #123 / Related to #123 line.
• Check one Workflow Source option in the PR body.
• Add a hidden marker such as <!-- workflow-source:local_request -->, <!-- workflow-source:manual_remote -->, <!-- workflow-source:review_followup -->, <!-- workflow-source:sync_campaign -->, or <!-- workflow-source:dependabot -->.
• Add a workflow source label such as workflow:source-direct-pr, workflow:source-local-request, workflow:source-review-followup, workflow:source-sync, or workflow:no-automation.

Once a valid source is present, this warning will not be reposted.

[github-actions]

Provider Comparison Report

Provider Summary

Provider	Model	Verdict	Confidence	Summary
openai	gpt-5.6-terra	FAIL	97%	The actual workflow fix is technically sound for its narrow purpose: materializing gh label output before grep avoids the pipefail/SIGPIPE false-negative behavior, and the equivalent consumer templ...
anthropic	claude-sonnet-5	CONCERNS	55%	The code change itself is a sound, well-documented fix for a real bash pitfall (SIGPIPE + pipefail causing grep -q on a piped gh command to falsely take the negative branch), applied consistent...

📋 Full Provider Details (click to expand)

openai

• Model: gpt-5.6-terra
• Verdict: FAIL
• Confidence: 97%
• Scores:
  • Correctness: 9.0/10
  • Completeness: 1.0/10
  • Quality: 8.0/10
  • Testing: 3.0/10
  • Risks: 8.0/10
• Summary: The actual workflow fix is technically sound for its narrow purpose: materializing gh label output before grep avoids the pipefail/SIGPIPE false-negative behavior, and the equivalent consumer template is kept in sync. The added missing-validator warning is also useful. However, this PR does not address the documented acceptance criteria it is being evaluated against, which concern CLI-agent task prompting, comment reduction, state/summary behavior, and agent-type-aware metadata handling. Therefore it fails on completeness despite the correctness of the implemented format-guard fix.
• Concerns:
  • The changes do not implement the documented acceptance criteria for CLI-agent PR behavior. They do not add explicit task appendices to Codex prompts, keepalive iteration summaries, or workflow step-summary output.
  • No changes implement the required CLI-agent comment consolidation: suppression of gate-summary and per-round instruction comments, a single keepalive-loop summary comment containing state, or the ≤3 bot-comments goal.
  • The required agent_type detect-job output, propagation of agent type to metadata updates, conditional status rendering, and conditional gate-summary behavior in agents-pr-meta.yml are not changed.
  • No tests were added for the format-guard pipefail/SIGPIPE regression or for the stated CLI-agent acceptance criteria.
  • The workflow writes labels.txt and held.txt in the workspace without cleanup. This is low risk on ephemeral GitHub-hosted runners, but mktemp would better avoid possible collisions or accidental interaction with repository files.

anthropic

• Model: claude-sonnet-5
• Verdict: CONCERNS
• Confidence: 55%
• Scores:
  • Correctness: 8.0/10
  • Completeness: 3.0/10
  • Quality: 8.0/10
  • Testing: 2.0/10
  • Risks: 4.0/10
• Summary: The code change itself is a sound, well-documented fix for a real bash pitfall (SIGPIPE + pipefail causing grep -q on a piped gh command to falsely take the negative branch), applied consistently in both the main and template copies of agents-issue-format-guard.yml, plus an added warning when the validator script is missing. However, this PR's scope and diff do not correspond to the acceptance criteria listed in the supplied planning context (CLI-agent task-appendix prompts, step-summary output in agents-keepalive-loop.yml, buildStatusBlock() agentType parameter, comment-count reduction) — those items remain unaddressed by this PR's code. No tests were added to validate either the format-guard fix or any of the documented ACs. Given the clear mismatch between the documented acceptance criteria and the actual code delivered, completeness against the stated criteria is low despite the underlying fix being technically correct and low-risk.
• Concerns:
  • The PR's actual diff (title: 'fix(agents): stop the format guard inverting its own label checks') is a targeted bugfix for a pipefail/grep-inversion issue in agents-issue-format-guard.yml. It has no relation to the documented acceptance criteria in the provided context, which are about CLI-agent task appendices, step-summary output, buildStatusBlock() changes, and comment-count reduction (from PR fix: add models:read permission for GitHub Models API #463/chore(codex): bootstrap PR for issue #101 #103 work).
  • None of the listed acceptance criteria ('CLI agent receives explicit tasks in prompt', '≤3 bot comments total', 'buildStatusBlock() accepts agentType', etc.) are addressed by this diff at all — the files changed (agents-issue-format-guard.yml, its template copy, and a langsmith JSON artifact) are unrelated to agents_pr_meta_update_body.js, agents-keepalive-loop.yml, or agents-pr-meta.yml mentioned in the tasks.
  • No test changes accompany the fix; the pipefail/grep-inversion bug fix is plausible and well-reasoned via inline comments, but there's no regression test proving the previous behavior was broken or that the fix resolves it.
  • The langsmith-fleet-worker-attempt.json diff (+2/-2) is unexplained in the PR description — unclear if it's an artifact leak or intentional state file change, which is a minor risk/quality concern.
  • The duplicated fix logic (materialize to labels.txt / held.txt) is copy-pasted across three call sites and two files (main + template), increasing maintenance burden; a shared helper function would have improved quality.

Agreement

• Correctness: scores within 1 point (avg 8.5/10, range 8.0-9.0)
• Quality: scores within 1 point (avg 8.0/10, range 8.0-8.0)
• Testing: scores within 1 point (avg 2.5/10, range 2.0-3.0)

Disagreement

Dimension	openai	anthropic
Verdict	FAIL	CONCERNS
Completeness	1.0/10	3.0/10
Risks	8.0/10	4.0/10

Unique Insights

• openai: The changes do not implement the documented acceptance criteria for CLI-agent PR behavior. They do not add explicit task appendices to Codex prompts, keepalive iteration summaries, or workflow step-summary output.; No changes implement the required CLI-agent comment consolidation: suppression of gate-summary and per-round instruction comments, a single keepalive-loop summary comment containing state, or the ≤3 bot-comments goal.; The required agent_type detect-job output, propagation of agent type to metadata updates, conditional status rendering, and conditional gate-summary behavior in agents-pr-meta.yml are not changed.; No tests were added for the format-guard pipefail/SIGPIPE regression or for the stated CLI-agent acceptance criteria.; The workflow writes labels.txt and held.txt in the workspace without cleanup. This is low risk on ephemeral GitHub-hosted runners, but mktemp would better avoid possible collisions or accidental interaction with repository files.
• anthropic: The PR's actual diff (title: 'fix(agents): stop the format guard inverting its own label checks') is a targeted bugfix for a pipefail/grep-inversion issue in agents-issue-format-guard.yml. It has no relation to the documented acceptance criteria in the provided context, which are about CLI-agent task appendices, step-summary output, buildStatusBlock() changes, and comment-count reduction (from PR fix: add models:read permission for GitHub Models API #463/chore(codex): bootstrap PR for issue #101 #103 work).; None of the listed acceptance criteria ('CLI agent receives explicit tasks in prompt', '≤3 bot comments total', 'buildStatusBlock() accepts agentType', etc.) are addressed by this diff at all — the files changed (agents-issue-format-guard.yml, its template copy, and a langsmith JSON artifact) are unrelated to agents_pr_meta_update_body.js, agents-keepalive-loop.yml, or agents-pr-meta.yml mentioned in the tasks.; No test changes accompany the fix; the pipefail/grep-inversion bug fix is plausible and well-reasoned via inline comments, but there's no regression test proving the previous behavior was broken or that the fix resolves it.; The langsmith-fleet-worker-attempt.json diff (+2/-2) is unexplained in the PR description — unclear if it's an artifact leak or intentional state file change, which is a minor risk/quality concern.; The duplicated fix logic (materialize to labels.txt / held.txt) is copy-pasted across three call sites and two files (main + template), increasing maintenance burden; a shared helper function would have improved quality.

🔍 LangSmith Traces

• openai
• anthropic

[stranske]

Closer verifier disposition (wrong-AC / false completeness FAIL)

Provider report on merged PR #2994 (comment 5229804806): openai FAIL (97%) / anthropic CONCERNS (55%).

Audited against the landed diff and PR title/body (not the stale #463 CLI-agent planning context the evaluators were given):

1. Scope match — Diff is exactly the format-guard pipefail/SIGPIPE fix (agents-issue-format-guard.yml root + consumer template): materialise gh … --jq label lists to files before grep -qx under set -o pipefail, plus a loud ::warning when the validator script is absent. Both providers explicitly call this technically sound.
2. Wrong acceptance criteria — Completeness FAIL/CONCERNS cite CLI-agent task appendices, keepalive step-summary, buildStatusBlock(agentType), comment consolidation, and agents-pr-meta.yml changes from old fix: add models:read permission for GitHub Models API #463/chore(codex): bootstrap PR for issue #101 #103 work. Those files are not in this PR's scope; the PR body no longer claims Closes #464 (already closed). Evaluating this narrow guard fix against that checklist is a false-negative completeness gate.
3. Residual notes (non-blocking) — Missing dedicated pipefail regression test and mktemp cleanup for labels.txt/held.txt are quality nits on ephemeral runners; no source issue remains open, so no follow-up PR is required for disposition.

Disposition: accept as delivered for the actual pipefail/guard scope; treat provider FAIL/CONCERNS as false-positive against wrong ACs. No source issue to close. No bounded follow-up opened.