chore: sync workflow templates - #292
Conversation
Automated sync from stranske/Workflows Template hash: d30555b90dd4 Changes synced from sync-manifest.yml
stranske
added
sync
🤖 Keepalive Loop StatusPR #292 | Agent: Codex | Iteration 0/5 Current State
🔍 Failure Classification| Error type | infrastructure | |
Keepalive Work Log (click to expand)
|
There was a problem hiding this comment.
Pull request overview
Syncs workflow templates from stranske/Workflows, expanding the weekly metrics pipeline with a download/unzip manifest contract and richer telemetry (terminal disposition + verifier model compatibility, bot-comment auth evidence), plus a few workflow/script robustness tweaks.
Changes:
- Add artifact download manifest generation/updates in
agents-weekly-metricsand emit a JSON summary contract alongside the markdown report. - Enrich metrics aggregation and terminal-disposition coverage with artifact/source attribution and verifier model compatibility checks.
- Extend bot-comment handler + auth coverage telemetry and refine keepalive actionable checklist counting.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/aggregate_agent_metrics.py |
Adds metric source attribution, parse-error details, artifact download contract ingestion, and JSON summary contract output. |
.github/workflows/agents-weekly-metrics.yml |
Initializes/records/finalizes artifact download manifest during downloads; uploads JSON summary + manifest artifacts. |
.github/workflows/agents-bot-comment-handler.yml |
Exposes skip_reason output and uploads a wrapper terminal disposition artifact for observability. |
.github/workflows/agents-81-gate-followups.yml |
Uses compact jq output (jq -cn) for emitted keepalive metrics JSON. |
.github/scripts/weekly_metrics_download_manifest.js |
New CLI to create/update/finalize a machine-readable download/unzip manifest plus markdown rendering. |
.github/scripts/weekly_metrics_artifacts.js |
Extends selection report with missing/priority family status summaries (used by downstream coverage tooling). |
.github/scripts/terminal_disposition_coverage.js |
Adds verifier model compatibility reporting and richer artifact selection normalization/markdown output. |
.github/scripts/terminal_disposition.js |
Extends terminal disposition normalization to include model + selection metadata fields. |
.github/scripts/keepalive_loop.js |
Introduces “actionable” checkbox counting (filters status/placeholder items) and adjusts reconciliation triggering. |
.github/scripts/coverage_monitor_summary.js |
New weekly coverage monitor summary contract combining terminal + bot-auth monitor outputs. |
.github/scripts/bot_comment_auth_coverage.js |
Tracks reusable invocation expectations/reasons and reports skipped vs missing organic evidence requirements. |
.github/scripts/agents_pr_meta_update_body.js |
Updates PR preamble generation to treat campaign issues differently (adds metadata marker + avoids “Closes” for campaign issues). |
| status: selectedCount > 0 ? 'selected' : (candidateCount > 0 ? 'available' : 'missing'), | ||
| candidate_count: candidateCount, | ||
| selected_count: selectedCount, | ||
| latest_candidate: selectedArtifact ? normalizeSelectionArtifact(selectedArtifact) : null, |
There was a problem hiding this comment.
In the fallback path (when priority_family_statuses is missing), latest_candidate is set to the selected artifact. This makes latest_candidate duplicate selected_artifact and misrepresents the “latest candidate” concept when selection metadata is incomplete. Consider leaving latest_candidate null (or deriving it from candidate artifacts if available) and only populating selected_artifact from selectedArtifacts.
| latest_candidate: selectedArtifact ? normalizeSelectionArtifact(selectedArtifact) : null, | |
| latest_candidate: null, |
| const STATUS_METRIC_LABELS = new Set([ | ||
| 'updated', | ||
| 'repos checked', | ||
| 'open sync prs', | ||
| 'open dependabot prs', | ||
| 'active review threads queued', | ||
| 'items needing local codex', | ||
| 'actionable local codex items', | ||
| 'claimable local codex items', | ||
| 'source-fixed candidates', | ||
| 'superseded sync candidates', | ||
| 'finished local results without published source changes', | ||
| 'claimed local codex items', | ||
| 'next claim lease expires', | ||
| ]); | ||
|
|
||
| function normaliseChecklistText(value) { | ||
| return normaliseTaskText(value) | ||
| .replace(/`([^`]+)`/g, '$1') | ||
| .replace(/[*_~]/g, '') | ||
| .trim() | ||
| .toLowerCase(); | ||
| } | ||
|
|
||
| function isStatusMetricChecklistItem(text) { | ||
| const normalized = normaliseChecklistText(text); | ||
| if (!normalized || !normalized.includes(':')) { | ||
| return false; | ||
| } | ||
| const [labelRaw, ...valueParts] = normalized.split(':'); | ||
| const label = labelRaw.trim(); | ||
| const value = valueParts.join(':').trim(); | ||
| if (!label || !value) { | ||
| return false; | ||
| } | ||
| if (!STATUS_METRIC_LABELS.has(label)) { | ||
| return false; | ||
| } | ||
| return true; | ||
| } | ||
|
|
||
| function isPlaceholderChecklistItem(text) { | ||
| const normalized = normaliseChecklistText(text); | ||
| if (!normalized) { | ||
| return true; | ||
| } | ||
| if (normalized.includes('section missing from source issue')) { | ||
| return true; | ||
| } | ||
| if (/^no tasks defined\.?$/.test(normalized)) { | ||
| return true; | ||
| } | ||
| if (/^no acceptance criteria defined\.?$/.test(normalized)) { | ||
| return true; | ||
| } | ||
| return false; | ||
| } | ||
|
|
||
| function isActionableChecklistItemText(text) { | ||
| return !isStatusMetricChecklistItem(text) && !isPlaceholderChecklistItem(text); | ||
| } | ||
|
|
||
| function toActionableChecklistCounts(markdown) { | ||
| const actionable = extractChecklistItems(markdown).filter((item) => isActionableChecklistItemText(item.text)); | ||
| const checked = actionable.filter((item) => item.checked).length; | ||
| const total = actionable.length; | ||
| return { | ||
| total, | ||
| checked, | ||
| unchecked: Math.max(0, total - checked), | ||
| }; |
There was a problem hiding this comment.
The new checklist filtering/counting logic (STATUS_METRIC_LABELS, placeholder detection, and toActionableChecklistCounts) changes how keepalive determines remaining work and when to flag reconciliation, but there are no unit tests covering these new rules. Add tests that verify status-metric and placeholder checkboxes are excluded and that only actionable items affect totals/unchecked counts.
| if (artifactDir) { | ||
| artifact.artifact_dir = artifactDir; | ||
| artifact.unzip.path = artifactDir; | ||
| } |
There was a problem hiding this comment.
updateArtifactResult assigns artifact.unzip.path before ensuring artifact.unzip exists. If a manifest is missing the unzip object (e.g., older manifest or manually edited), this will throw a TypeError and prevent recording download results. Initialize artifact.unzip (and artifact.download) to an object when absent before mutating nested fields.
|
Superseded by merged latest Workflows sync PR #324 from branch sync/workflows-3e5a27aaceee. Closing this older sync/workflows-* lane without deleting the branch. |
Sync Summary
Files Updated
Files Skipped
Review Checklist
Source: stranske/Workflows
Manifest:
.github/sync-manifest.yml