Skip to content

fix: add token-aware retry to issue workflows (#1173)#1173

Merged
stranske merged 9 commits intomainfrom
fix/load-balancer-remaining
Feb 1, 2026
Merged

fix: add token-aware retry to issue workflows (#1173)#1173
stranske merged 9 commits intomainfrom
fix/load-balancer-remaining

Conversation

@stranske
Copy link
Copy Markdown
Owner

@stranske stranske commented Feb 1, 2026

Source: Issue #123

Automated Status Summary

Scope

After merging PR #103 (multi-agent routing infrastructure), we need to:

  1. Validate the CLI agent pipeline works end-to-end with the new task-focused prompts
  2. Add GITHUB_STEP_SUMMARY output so iteration results are visible in the Actions UI
  3. Streamline the Automated Status Summary to reduce clutter when using CLI agents
  4. Clean up comment patterns to avoid a mix of old UI-agent and new CLI-agent comments

Context for Agent

Design Decisions & Constraints

    1. Clean up comment patterns to avoid a mix of old UI-agent and new CLI-agent comments
  • The keepalive loop now:
  • | <!-- keepalive-loop-summary --> | github-actions[bot] | NEW: CLI agent iteration tracking | ✅ Keep for CLI agents |
  • | <!-- keepalive-state:v1 --> | agents-workflows-bot[bot] | State tracking | ⚠️ Multiple copies accumulate |
  • | <!-- keepalive-round: N --> | stranske | OLD: Instruction comment | ❌ CLI agents dont need this |
  • The goal: For CLI agents (agent:* label), we should have exactly one updating comment (<!-- keepalive-loop-summary -->) instead of accumulating 10+ comments per PR.
  • Requires PR #103 to be merged first
  • This round you MUST:
  • Review the Scope/Tasks/Acceptance below, identify the next incomplete task that requires code, implement it, then post a reply comment with the completed items using their exact original text.

Related Issues/PRs

References

Blockers & Dependencies

  • After merging PR #103 (multi-agent routing infrastructure), we need to:
    1. Mark a task checkbox complete ONLY after verifying the implementation works.

Tasks

Pipeline Validation

  • After PR chore(codex): bootstrap PR for issue #101 #103 merges, create a test PR with agent:codex label
  • Verify task appendix appears in Codex prompt (check workflow logs)
  • Verify Codex works on actual tasks (not random infrastructure work)
  • Verify keepalive comment updates with iteration progress

GITHUB_STEP_SUMMARY

  • Add step summary output to agents-keepalive-loop.yml after agent run
  • Include: iteration number, tasks completed, files changed, outcome
  • Ensure summary is visible in workflow run UI

Conditional Status Summary

  • Modify buildStatusBlock() in agents_pr_meta_update_body.js to accept agentType parameter
  • When agentType is set (CLI agent): hide workflow table, hide head SHA/required checks
  • Keep Scope/Tasks/Acceptance checkboxes for all cases
  • Pass agent type from workflow to the update_body job

Comment Pattern Cleanup

  • For CLI agents (agent:* label):
  • Suppress <!-- gate-summary: --> comment posting (use step summary instead)
  • Suppress <!-- keepalive-round: N --> instruction comments (task appendix replaces this)
  • Update <!-- keepalive-loop-summary --> to be the single source of truth
  • Ensure state marker is embedded in the summary comment (not separate)
  • For UI Codex (no agent:* label):
  • Keep existing comment patterns (instruction comments, connector bot reports)
  • Keep <!-- gate-summary: --> comment
  • Add agent_type output to detect job so downstream workflows know the mode
  • Update agents-pr-meta.yml to conditionally skip gate summary for CLI agent PRs

Acceptance criteria

  • CLI agent receives explicit tasks in prompt and works on them
  • Iteration results visible in Actions workflow run summary
  • PR body shows checkboxes but not workflow clutter when using CLI agents
  • UI Codex path (no agent label) continues to show full status summary
  • CLI agent PRs have ≤3 bot comments total (summary, one per iteration update) instead of 10+
  • State tracking is consolidated in the summary comment, not scattered

Head SHA: 90cea92
Latest Runs: ✅ success — Gate
Required: gate: ✅ success

Workflow / Job Result Logs
Agents PR meta manager ❔ in progress View run
CI Autofix Loop ✅ success View run
Gate ✅ success View run
Health 40 Sweep ✅ success View run
Health 44 Gate Branch Protection ✅ success View run
Health 45 Agents Guard ✅ success View run
Health 50 Security Scan ✅ success View run
Health 73 Template Completeness ✅ success View run
Maint 52 Validate Workflows ✅ success View run
PR 11 - Minimal invariant CI ✅ success View run
Selftest CI ✅ success View run
Validate Sync Manifest ✅ success View run

Copilot AI review requested due to automatic review settings February 1, 2026 02:58
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c4fb47417a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

},
{
}),
}),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Remove stray closing brace that breaks the test file

This extra }), closes the withEmptyJobs({ ... }) call twice, leaving an unmatched ) in the array literal. That makes the entire test file invalid JavaScript, so any run of this test suite (or any runner that loads the file) will fail to parse before tests execute.

Useful? React with 👍 / 👎.

@stranske stranske force-pushed the fix/load-balancer-remaining branch from c4fb474 to efbdeb2 Compare February 1, 2026 03:01
@agents-workflows-bot
Copy link
Copy Markdown
Contributor

⚠️ Action Required: Unable to determine source issue for PR #1173. The PR title, branch name, or body must contain the issue number (e.g. #123, branch: issue-123, or the hidden marker ).

@stranske stranske temporarily deployed to agent-high-privilege February 1, 2026 03:01 — with GitHub Actions Inactive
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR wires token-aware GitHub API retries and load-balanced tokens into several workflows and utilities, improving resilience to rate limits while extending issue/PR metadata handling and verifier reporting.

Changes:

  • Enhance scripts/check_issue_consistency.py to infer issue numbers from the head ref, recognize autofix contexts, and restrict header scanning to comments/docstrings; add corresponding tests.
  • Replace gh CLI issue calls with Octokit-based, token-aware retry flows in agents workflows (issue optimizer, auto-pilot, issue intake) and gate/autofix paths, including new Node/Octokit setup and load balancer token exports.
  • Extend verifier CI query behavior/tests to include per-workflow jobs summaries and error fields, and update agents verifier context tests to match.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
verifier-pr-diff.patch Fixture patch updated; no functional impact on workflows or scripts.
tests/scripts/test_issue_consistency_check.py Adjust test to treat only comment lines as header issue references and add coverage for extracting issue numbers from the head ref branch name.
scripts/check_issue_consistency.py Add extract_head_ref_issue_numbers, is_autofix_context, more precise header scanning over comments/docstrings, and head-ref/autofix-based fallback logic in main.
.github/workflows/pr-00-gate.yml Introduce load balancer token export, Node/Octokit setup, and token-aware retry/pagination for gate autofix labeling and commit-status posting.
.github/workflows/autofix.yml Switch context resolution to use createTokenAwareRetry/paginateWithRetry and ensure load balancer tokens and Node/Octokit deps are available.
.github/workflows/agents-issue-optimizer.yml Replace gh issue API usage with Octokit + token-aware retry for reading/updating issues and comments; add Node/Octokit and token load balancer setup.
.github/workflows/agents-bot-comment-handler.yml Make the resolver job use token-aware retry when available (falling back to existing retry helpers) for repo/PR lookups.
.github/workflows/agents-auto-pilot.yml Introduce Node/Octokit + token-aware retry for resolving external workflow refs, fetching keepalive state, and performing inline issue formatting/optimization/apply steps via the GitHub API instead of gh.
.github/workflows/agents-63-issue-intake.yml Add Node/Octokit and load balancer setup, and convert LangChain-based mass formatting to use token-aware Octokit for issue reads/updates and labeling.
.github/scripts/__tests__/verifier-ci-query.test.js Update expectations to include job summary/error fields and introduce withEmptyJobs helper, but currently contains a syntax error in one assertion array.
.github/scripts/__tests__/agents-verifier-context.test.js Add withEmptyJobs helper and update CI results expectations to include empty jobs metadata, aligning with the updated verifier CI query output.
Comments suppressed due to low confidence (1)

.github/workflows/agents-63-issue-intake.yml:296

  • This second Export load balancer tokens step immediately repeats the same action that was run just above, but with a reduced set of inputs. Running the exporter twice back-to-back is redundant and may overwrite the more complete token configuration from the first call; consider consolidating to a single invocation with the intended inputs.
      - name: Export load balancer tokens
        uses: ./.github/actions/export-load-balancer-tokens
        with:
          github_token: ${{ github.token }}
          token_rotation_json: ${{ secrets.TOKEN_ROTATION_JSON }}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +569 to +573
cat > /tmp/auto_pilot_format_start.md <<'EOF'
🤖 **Auto-pilot step $((STEP_COUNT + 1))**: Starting issue formatting...

Running formatter inline (not via label trigger).
EOF
Copy link

Copilot AI Feb 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this here-document uses a single-quoted delimiter (<<'EOF'), the $((STEP_COUNT + 1)) expression inside will not be expanded and will appear literally in the comment body. To preserve the dynamic step number in the posted message, switch to an unquoted here-doc delimiter (or otherwise interpolate the step count before writing the file).

Copilot uses AI. Check for mistakes.
Comment on lines +767 to +771
cat > /tmp/auto_pilot_optimize_start.md <<'EOF'
🤖 **Auto-pilot step $((STEP_COUNT + 1))**: Analyzing issue for improvements...

Running optimizer inline (not via label trigger).
EOF
Copy link

Copilot AI Feb 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As with the format step, this here-document is single-quoted (<<'EOF'), so the $((STEP_COUNT + 1)) expression will not be interpolated and the literal text will be posted. If you intend to include the actual step number in the progress comment, the here-doc delimiter should be unquoted (or the step count should be substituted before writing the file).

Copilot uses AI. Check for mistakes.
Comment on lines +1048 to +1052
cat > /tmp/auto_pilot_apply_start.md <<'EOF'
🤖 **Auto-pilot step $((STEP_COUNT + 1))**: Applying optimization suggestions...

Running apply inline (not via label trigger).
EOF
Copy link

Copilot AI Feb 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This apply-step progress comment has the same quoting pattern (<<'EOF') as the earlier ones, so the $((STEP_COUNT + 1)) arithmetic expansion will not be evaluated and the literal placeholder will be written. To keep the user-facing comment consistent with the previous behavior (showing the actual step number), use an unquoted here-doc delimiter or otherwise interpolate STEP_COUNT before writing the file.

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 1, 2026

Automated Status Summary

Head SHA: 9463138
Latest Runs: ⏳ pending — Gate
Required contexts: Gate / gate, Health 45 Agents Guard / Enforce agents workflow protections
Required: core tests (3.11): ⏳ pending, core tests (3.12): ⏳ pending, docker smoke: ⏳ pending, gate: ⏳ pending

Workflow / Job Result Logs
(no jobs reported) ⏳ pending

Coverage Overview

  • Coverage history entries: 1

Coverage Trend

Metric Value
Current 93.12%
Baseline 85.00%
Delta +8.12%
Minimum 70.00%
Status ✅ Pass

Top Coverage Hotspots (lowest coverage)

File Coverage Missing
src/cli_parser.py 81.8% 4
src/percentile_calculator.py 95.0% 1
src/aggregator.py 95.0% 2
src/__init__.py 100.0% 0
src/ndjson_parser.py 100.0% 0

Updated automatically; will refresh on subsequent CI/Docker completions.


Keepalive checklist

Scope

After merging PR #103 (multi-agent routing infrastructure), we need to:

Context for Agent

Design Decisions & Constraints

    1. Clean up comment patterns to avoid a mix of old UI-agent and new CLI-agent comments
  • The keepalive loop now:
  • | <!-- keepalive-loop-summary --> | github-actions[bot] | NEW: CLI agent iteration tracking | ✅ Keep for CLI agents |
  • | <!-- keepalive-state:v1 --> | agents-workflows-bot[bot] | State tracking | ⚠️ Multiple copies accumulate |
  • | <!-- keepalive-round: N --> | stranske | OLD: Instruction comment | ❌ CLI agents dont need this |
  • The goal: For CLI agents (agent:* label), we should have exactly one updating comment (<!-- keepalive-loop-summary -->) instead of accumulating 10+ comments per PR.
  • Requires PR #103 to be merged first
  • This round you MUST:
  • Review the Scope/Tasks/Acceptance below, identify the next incomplete task that requires code, implement it, then post a reply comment with the completed items using their exact original text.
  • ⚠️ Action Required: Unable to determine source issue for PR #1173. The PR title, branch name, or body must contain the issue number (e.g. #123, branch: issue-123, or the hidden marker ).
  • ⚠️ Action Required: Unable to determine source issue for PR #1173. The PR title, branch name, or body must contain the issue number (e.g. #123, branch: issue-123, or the hidden marker ).
  • | Keepalive | ❌ disabled |

Related Issues/PRs

References

Blockers & Dependencies

  • After merging PR #103 (multi-agent routing infrastructure), we need to:
  • After merging PR #103 (multi-agent routing infrastructure), we need to:
    1. Mark a task checkbox complete ONLY after verifying the implementation works.

Tasks

  • 1. Validate the CLI agent pipeline works end-to-end with the new task-focused prompts
  • 2. Add GITHUB_STEP_SUMMARY output so iteration results are visible in the Actions UI
  • 3. Streamline the Automated Status Summary to reduce clutter when using CLI agents
  • 4. Clean up comment patterns to avoid a mix of old UI-agent and new CLI-agent comments

Acceptance criteria

  • CLI agent receives explicit tasks in prompt and works on them
  • Iteration results visible in Actions workflow run summary
  • PR body shows checkboxes but not workflow clutter when using CLI agents
  • UI Codex path (no agent label) continues to show full status summary
  • CLI agent PRs have ≤3 bot comments total (summary, one per iteration update) instead of 10+
  • State tracking is consolidated in the summary comment, not scattered
  • [ ]

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 1, 2026

🤖 Keepalive Loop Status

PR #1173 | Agent: Codex | Iteration 0/5

Current State

Metric Value
Iteration progress [----------] 0/5
Action wait (missing-agent-label)
Disposition skipped (transient)
Gate success
Tasks 0/27 complete
Timeout 45 min (default)
Timeout usage 2m elapsed (6%, 43m remaining)
Keepalive ❌ disabled
Autofix ❌ disabled

🔍 Failure Classification

| Error type | infrastructure |
| Error category | resource |
| Suggested recovery | Confirm the referenced resource exists (repo, PR, branch, workflow, or file). |

@stranske stranske temporarily deployed to agent-high-privilege February 1, 2026 03:05 — with GitHub Actions Inactive
@stranske stranske temporarily deployed to agent-high-privilege February 1, 2026 03:08 — with GitHub Actions Inactive
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 1, 2026

✅ Codex Completion Checkpoint

Commit: 90cea92
Recorded: 2026-02-01T03:16:23.091Z

No new completions recorded this round.

About this comment

This comment is automatically generated to track task completions.
The Automated Status Summary reads these checkboxes to update PR progress.
Do not edit this comment manually.

@github-actions github-actions bot added the autofix Opt-in automated formatting & lint remediation label Feb 1, 2026
@stranske stranske changed the title fix: add token-aware retry to issue workflows fix: add token-aware retry to issue workflows (#1173) Feb 1, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 1, 2026

Status | ✅ no new diagnostics
History points | 1
Timestamp | 2026-02-01 03:18:58 UTC
Report artifact | autofix-report-pr-1173
Remaining | 0
New | 0
No additional artifacts

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 1, 2026

Autofix updated these files:

  • scripts/check_issue_consistency.py

@stranske stranske merged commit bfc48e3 into main Feb 1, 2026
361 checks passed
@stranske stranske deleted the fix/load-balancer-remaining branch February 1, 2026 03:25
stranske added a commit that referenced this pull request Feb 1, 2026
* test: update verifier CI expectations

* fix: add token-aware retry to issue workflows

* fix: indent node heredocs

* fix: repair verifier ci query test

* chore(codex-autofix): apply updates (PR #1173)

* chore(autofix): formatting/lint

* fix: avoid heredoc in intake formatter

* fix: remove unused intake variable

* fix: wrap workflow API calls with retry

* docs: add high-volume API audit

* fix: extend retry coverage for remediation

* docs: refresh audit entries after remediation

* fix: wrap belt worker graphql with retry

* docs: update audit for gh api cleanup

* fix: complete retry and load balancer coverage

* fix: address PR review feedback

* chore: sync template scripts

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
stranske added a commit that referenced this pull request Feb 1, 2026
* test: update verifier CI expectations

* fix: add token-aware retry to issue workflows

* fix: indent node heredocs

* fix: repair verifier ci query test

* chore(codex-autofix): apply updates (PR #1173)

* chore(autofix): formatting/lint

* fix: avoid heredoc in intake formatter

* fix: remove unused intake variable

* fix: wrap workflow API calls with retry

* docs: add high-volume API audit

* fix: extend retry coverage for remediation

* docs: refresh audit entries after remediation

* fix: wrap belt worker graphql with retry

* docs: update audit for gh api cleanup

* fix: complete retry and load balancer coverage

* fix: address PR review feedback

* feat: add API guardrails and audits

* fix: resolve CI failures and agent feedback

* fix: correct autopilot YAML indentation

* chore(codex-autofix): apply updates (PR #1179)

* chore: sync template scripts

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autofix Opt-in automated formatting & lint remediation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants