chore: sync workflow templates - #606
Conversation
Automated sync from stranske/Workflows Template hash: daec38e5dbf4 Changes synced from sync-manifest.yml
🤖 Keepalive Loop StatusPR #606 | Agent: Codex | Iteration 0/5 Current State
🔍 Failure Classification| Error type | infrastructure | |
Keepalive Work Log (click to expand)
|
There was a problem hiding this comment.
Pull request overview
Synchronizes this repository’s workflow automation and metrics tooling with the upstream workflow templates, expanding weekly metrics collection/aggregation and adding richer “contract” artifacts for debugging and reporting.
Changes:
- Enhanced weekly metrics workflow to generate a download manifest (JSON/Markdown) while fetching artifacts, and to publish a JSON summary alongside the existing Markdown summary.
- Extended metrics aggregation and terminal-disposition coverage tooling to enrich records with source metadata and to track verifier model compatibility / missing model metadata.
- Updated the bot-comment handler wrapper workflow to emit and upload a wrapper terminal-disposition artifact (plus skip reasons) to support coverage/telemetry.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
scripts/aggregate_agent_metrics.py |
Adds metric source enrichment, parse error detail reporting, JSON “summary contract” output, and verifier model metadata reporting. |
.github/workflows/agents-weekly-metrics.yml |
Tracks artifact download/unzip outcomes via a manifest and publishes additional artifacts (JSON summary + manifest). |
.github/workflows/agents-bot-comment-handler.yml |
Emits wrapper terminal disposition records and exposes skip reasons as job outputs. |
.github/workflows/agents-81-gate-followups.yml |
Adjusts jq invocation to compact output when building metrics JSON. |
.github/scripts/weekly_metrics_download_manifest.js |
New helper to initialize/record/finalize the weekly metrics artifact download manifest (JSON + optional Markdown). |
.github/scripts/weekly_metrics_artifacts.js |
Adds priority-family “missing/available/selected” reporting to the artifact selection contract and Markdown output. |
.github/scripts/terminal_disposition_coverage.js |
Adds verifier model compatibility checks and enriches artifact selection summary with priority-family status info. |
.github/scripts/terminal_disposition.js |
Extends normalized terminal disposition records with model/mode metadata fields. |
.github/scripts/coverage_monitor_summary.js |
New weekly coverage-monitor summary contract that aggregates monitor report statuses into a single summary. |
.github/scripts/bot_comment_auth_coverage.js |
Extends organic evidence reporting with “skipped/missing requirements” details and wrapper→reusable expectations metadata. |
.github/scripts/agents_pr_meta_update_body.js |
Updates PR preamble generation to include an issue marker and an auto-close reference. |
| byFamily.set(family, { | ||
| family, | ||
| status: selectedCount > 0 ? 'selected' : (candidateCount > 0 ? 'available' : 'missing'), | ||
| candidate_count: candidateCount, | ||
| selected_count: selectedCount, | ||
| latest_candidate: selectedArtifact ? normalizeSelectionArtifact(selectedArtifact) : null, | ||
| selected_artifact: selectedArtifact ? normalizeSelectionArtifact(selectedArtifact) : null, |
There was a problem hiding this comment.
In normalizeTerminalPriorityFamilyStatuses, the fallback path for families not present in report.priority_family_statuses sets latest_candidate to the selected artifact (same object as selected_artifact). This misrepresents what “latest candidate” means and can lead to confusing/incorrect coverage markdown (the “Latest artifact” column will mirror the selected artifact even when there are no candidate details). Consider leaving latest_candidate as null in this fallback (or deriving it from candidate artifacts if available) and only populating selected_artifact from the selected list.
| byFamily.set(family, { | |
| family, | |
| status: selectedCount > 0 ? 'selected' : (candidateCount > 0 ? 'available' : 'missing'), | |
| candidate_count: candidateCount, | |
| selected_count: selectedCount, | |
| latest_candidate: selectedArtifact ? normalizeSelectionArtifact(selectedArtifact) : null, | |
| selected_artifact: selectedArtifact ? normalizeSelectionArtifact(selectedArtifact) : null, | |
| const normalizedSelectedArtifact = selectedArtifact | |
| ? normalizeSelectionArtifact(selectedArtifact) | |
| : null; | |
| byFamily.set(family, { | |
| family, | |
| status: selectedCount > 0 ? 'selected' : (candidateCount > 0 ? 'available' : 'missing'), | |
| candidate_count: candidateCount, | |
| selected_count: selectedCount, | |
| latest_candidate: null, | |
| selected_artifact: normalizedSelectedArtifact, |
Sync Summary
Files Updated
Files Skipped
Review Checklist
Source: stranske/Workflows
Manifest:
.github/sync-manifest.yml