chore: sync workflow templates - #598
Conversation
Automated sync from stranske/Workflows Template hash: 49bfb75a2f6f Changes synced from sync-manifest.yml
🤖 Keepalive Loop StatusPR #598 | Agent: Codex | Iteration 0/5 Current State
🔍 Failure Classification| Error type | infrastructure | |
Keepalive Work Log (click to expand)
|
There was a problem hiding this comment.
Pull request overview
Syncs workflow-template updates from stranske/Workflows, expanding weekly metrics/coverage telemetry and adding wrapper terminal-disposition reporting for the deprecated bot-comment handler workflow.
Changes:
- Extend agent metrics aggregation and reporting to include verifier model + selection metadata.
- Add artifact-selection diagnostics (priority family missing/status) and propagate them into terminal disposition coverage output.
- Add wrapper terminal-disposition artifact emission for
agents-bot-comment-handlerruns; introduce a new coverage monitor summary script and enhance bot-comment auth coverage reporting.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/aggregate_agent_metrics.py |
Adds model / selection-reason / verifier-mode counters to the verifier summary and prints them in the markdown summary. |
.github/workflows/agents-bot-comment-handler.yml |
Emits a wrapper terminal disposition artifact and threads through skip reasons for better telemetry. |
.github/scripts/weekly_metrics_artifacts.js |
Adds “missing priority families” + per-family status reporting to the metrics artifact selector report/markdown. |
.github/scripts/terminal_disposition_coverage.js |
Normalizes and renders priority-family selection status in terminal disposition coverage reports. |
.github/scripts/terminal_disposition.js |
Extends terminal disposition normalization to include LLM model / selection reason / verifier mode fields. |
.github/scripts/coverage_monitor_summary.js |
Adds a new summary contract to roll up coverage monitor reports into a single JSON/markdown output. |
.github/scripts/bot_comment_auth_coverage.js |
Tracks reusable-invocation expectations and annotates organic evidence requirements as skipped/missing. |
| - name: Upload wrapper terminal disposition | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: review-thread-terminal-disposition-${{ github.run_id }} | ||
| path: | | ||
| agent-metrics/review-thread-terminal-disposition.ndjson | ||
| terminal-disposition-summary.md | ||
| if-no-files-found: error | ||
| retention-days: 14 |
There was a problem hiding this comment.
actions/upload-artifact is referenced by the floating @v7 tag here, while the rest of the repo pins third-party actions to a full commit SHA (with an inline major-version comment). Please pin this to the same upload-artifact v7 SHA used elsewhere in workflows (e.g. agents-weekly-metrics.yml, agents-keepalive-loop.yml) to avoid supply-chain drift.
| - name: Write wrapper terminal disposition | ||
| if: always() | ||
| env: | ||
| RESOLVED_PR_NUMBER: ${{ steps.resolve.outputs.pr_number }} | ||
| REUSABLE_INVOCATION_EXPECTED: ${{ steps.resolve.outputs.should_run }} | ||
| SKIP_REASON: ${{ steps.resolve.outputs.skip_reason }} | ||
| run: | | ||
| mkdir -p agent-metrics | ||
| node <<'NODE' | ||
| const fs = require('fs'); | ||
| const helperPath = './.github/scripts/terminal_disposition.js'; |
There was a problem hiding this comment.
This job runs Node.js directly (the heredoc node invocation) and also uses ./.github/actions/setup-api-client (which calls node/npm internally), but there is no explicit actions/setup-node step. Other workflows in this repo consistently pin actions/setup-node (Node 20) before setup-api-client; add the same here so the workflow doesn’t depend on whatever Node version happens to be preinstalled on ubuntu-latest.
| status: selectedCount > 0 ? 'selected' : (candidateCount > 0 ? 'available' : 'missing'), | ||
| candidate_count: candidateCount, | ||
| selected_count: selectedCount, | ||
| latest_candidate: selectedArtifact ? normalizeSelectionArtifact(selectedArtifact) : null, |
There was a problem hiding this comment.
In the fallback path (when report.priority_family_statuses doesn’t include a family), latest_candidate is being set from selectedArtifact. That conflates “latest available candidate” with “selected artifact” and can make the coverage markdown/table misleading. Consider leaving latest_candidate as null here (or populating it from actual candidate artifacts if you have them) and keep selected_artifact as the selected one.
| latest_candidate: selectedArtifact ? normalizeSelectionArtifact(selectedArtifact) : null, | |
| latest_candidate: null, |
Sync Summary
Files Updated
Files Skipped
Review Checklist
Source: stranske/Workflows
Manifest:
.github/sync-manifest.yml