Conversation
Automated sync from stranske/Workflows Template hash: 164e3ba9e677 Changes synced from sync-manifest.yml
🤖 Keepalive Loop StatusPR #914 | Agent: Codex | Iteration 0/5 Current State
🔍 Failure Classification| Error type | infrastructure | |
Keepalive Work Log (click to expand)
|
There was a problem hiding this comment.
Pull request overview
Syncs workflow-template updates from stranske/Workflows, expanding weekly metrics/coverage telemetry and adding more detailed terminal-disposition and bot-auth reporting to support the repo’s agents automation.
Changes:
- Add wrapper-side terminal disposition artifact emission to the bot comment handler workflow.
- Extend weekly metrics artifact selection + terminal disposition coverage reporting with priority-family statuses and verifier model compatibility signals.
- Enhance metrics aggregation/reporting to include verifier model/mode and model-selection metadata summaries.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/aggregate_agent_metrics.py |
Adds verifier-model/mode and model-selection metadata counters to the aggregated markdown summary. |
.github/workflows/agents-bot-comment-handler.yml |
Emits and uploads a wrapper “review-thread terminal disposition” artifact and exposes skip_reason output. |
.github/scripts/weekly_metrics_artifacts.js |
Adds missing-priority-family and per-family status reporting to the artifact selection contract/markdown. |
.github/scripts/terminal_disposition_coverage.js |
Adds verifier model compatibility checks and richer selection-family normalization/markdown output. |
.github/scripts/terminal_disposition.js |
Extends terminal disposition normalization to carry model/mode metadata fields. |
.github/scripts/coverage_monitor_summary.js |
Introduces a weekly “coverage monitor summary” contract (terminal + bot-auth reports). |
.github/scripts/bot_comment_auth_coverage.js |
Improves organic evidence reporting with skipped/missing requirement details tied to wrapper expectations. |
| - name: Upload wrapper terminal disposition | ||
| if: always() | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: review-thread-terminal-disposition-${{ github.run_id }} | ||
| path: | | ||
| agent-metrics/review-thread-terminal-disposition.ndjson | ||
| terminal-disposition-summary.md | ||
| if-no-files-found: error | ||
| retention-days: 14 |
There was a problem hiding this comment.
actions/upload-artifact is referenced as a floating @v7 tag here, while other workflows in this repo pin upload-artifact v7 to a specific commit SHA. To keep the supply-chain posture consistent (and avoid unexpected action updates), pin this step to the same SHA used elsewhere in .github/workflows/.
| elif is_terminal_disposition: | ||
| verifier_mode = str(entry.get("verifier_mode") or "").strip().lower() | ||
| if verifier_mode != "evaluate": | ||
| disposition = entry.get("disposition") or entry.get("terminal_state") or "unknown" | ||
| missing_verifier_model_metadata[str(disposition)] += 1 | ||
| model_selection_reason = entry.get("codex_model_selection_reason") or entry.get( |
There was a problem hiding this comment.
The new missing_verifier_model_metadata counter will include all terminal disposition records that lack a model and are not verifier_mode == evaluate. With the newly added review-thread terminal disposition records (wrapper artifacts), this will start flagging non-verifier terminal dispositions as “missing verifier model metadata”, skewing the verifier summary. Consider restricting this check to verifier terminal disposition records only (e.g., artifact_family == 'verifier-terminal-disposition' or another verifier-specific signal).
|
|
||
| function overallStatus(monitors) { | ||
| if (monitors.some((monitor) => monitor.should_fail || monitor.status === 'fail')) return 'fail'; | ||
| if (monitors.some((monitor) => ['missing', 'parse-error', 'warning'].includes(monitor.status))) { |
There was a problem hiding this comment.
normalizeStatus() can return unknown, but overallStatus() doesn’t treat unknown as a warning/failure condition. This can allow a malformed/unsupported report status to still yield an overall pass. Consider handling unknown explicitly (e.g., treat as warning) so unexpected report formats can’t silently pass the monitor.
| if (monitors.some((monitor) => ['missing', 'parse-error', 'warning'].includes(monitor.status))) { | |
| if (monitors.some((monitor) => ['missing', 'parse-error', 'warning', 'unknown'].includes(monitor.status))) { |
Sync Summary
Files Updated
Files Skipped
Review Checklist
Source: stranske/Workflows
Manifest:
.github/sync-manifest.yml