π₯ Daily Health Check β 2026-08-08
Status: π΄ 2 critical Β· π‘ 2 warnings Β· π΅ 0 info
Since yesterday: π 2 new Β· β
1 resolved Β· π 2 unchanged
π New Findings (2)
These appeared since the last health check (2026-08-06).
π΄ Evaluation workflow failed on main: "Select available Copilot token from pool" step
- Fingerprint:
pipeline:evaluation:evaluate-/-vally-(dotnet-template-engine):select-available-copilot-token-from-pool:failure
- Details: Run #31229776756 (scheduled run on
main, started 2026-08-08T00:19:38Z) failed in job evaluate / vally (dotnet-template-engine) at the Select available Copilot token from pool step. The run took an unusually long 180.85 minutes before failing/completing (vs. a typical ~20β60 min), suggesting the token-pool selection logic stalled or exhausted retries rather than failing fast. This directly correlates with the eval-duration critical finding below β the same run is the primary driver of today's inflated 14-day average.
- Action: Inspect the Copilot PAT pool health (see the related Validate PAT Pool warning) and the token-selection retry/backoff logic in the evaluation workflow. If the pool is exhausted or a token is misbehaving, the selection step may be looping/waiting instead of failing quickly.
π΄ Evaluation average duration exceeds 55-minute critical threshold
- Fingerprint:
resource:eval-duration:critical
- Details: Average
evaluation.yml run duration over the last 14 days (main branch, 24 completed runs) is 57.7 minutes, above the 55-minute critical threshold (and well above the 50-minute warning threshold). This is skewed by the 180.85-minute outlier run above; excluding it, most other runs in the sample ranged ~18β70 minutes, several already exceeding the 50-minute warning band.
- Action: Investigate whether the stalled token-selection step (see above finding) is inflating the average, and separately evaluate whether the evaluation job needs performance optimization or an increased timeout/schedule interval independent of that incident.
π Investigation Results
Deep investigations are dispatched for new critical/warning findings.
The grooming workflow links results ~3 hours after this run.
β
Resolved Since Yesterday (1)
These were in yesterday's report but are no longer detected.
Evaluation workflow run cancelled (deploy-dashboard job)
- Fingerprint:
pipeline:evaluation:deploy-dashboard:timeout
- Resolution: No matching cancelled-run finding was detected in the current 24h window on
main β no cancelled runs occurred. This appears to have been an isolated single occurrence, as previously assessed.
π Existing Findings (2)
These have been present since before today. Sorted by age.
π‘ Warning β Orphan plugin: dotnet-experimental not in marketplace.json Β· first seen 2026-05-14 Β· 62 occurrences
Fingerprint: infra:orphan-plugin:dotnet-experimental
Category: Infrastructure Β· Severity: π‘ Warning
The plugin directory plugins/dotnet-experimental/ has a valid plugin.json but is not listed in .github/plugin/marketplace.json. Consumers cannot discover this plugin. 86 days outstanding.
Links: plugin.json Β· marketplace.json
Suggested action: Either add dotnet-experimental to marketplace.json if it is ready for consumers, or remove the plugin directory if it is no longer needed.
π‘ Warning β Validate PAT Pool workflow failed: Build summary step Β· first seen 2026-08-05 Β· 3 occurrences
Fingerprint: pipeline:validate-pat-pool:validate-copilot-pat-pool:build-summary:failure
Category: Pipeline Β· Severity: π‘ Warning
The Validate PAT Pool workflow's Validate Copilot PAT Pool job failed again at the Build summary step on today's scheduled run (Run #31236108858), the third consecutive occurrence. All 10 individual Validate COPILOT_PAT_N steps succeeded β only the summary-generation step fails, continuing to look like a scripting/formatting bug rather than an actual PAT pool health problem.
Links: Run #31236108858
Suggested action: Investigate the summary-generation logic in validate-pat-pool.yml's "Build summary" step; three consecutive failures strongly suggest a persistent bug rather than a transient issue β worth prioritizing a fix.
π Trends (7-day)
| Metric |
Today |
7d Avg |
Ξ |
Trend |
| Eval duration (min) |
57.7 |
~23β58 |
β |
β οΈ |
| Eval success rate (main) |
100% (23/24 non-cancelled completed, 14d; 1 failure) |
~100% |
~0 |
β‘οΈ |
| Eval success rate (all branches, 24h) |
90.6% (3 failures / 32 non-cancelled, 60 total incl. skipped) |
~88% |
+2.6pp |
β
|
| Eval scheduled cancellation rate |
0% (0/1 scheduled run, main, 24h) |
~0% |
0 |
β
|
| Workflow failure rate (7d) |
2 fingerprints observed (Validate PAT Pool "Build summary"; Evaluation token-pool step) |
N/A |
β |
β οΈ |
| Compute hours/day |
N/A (not computed this run) |
N/A |
β |
β‘οΈ |
Note: Eval failure rate across all branches (3/32 = 9.4%) remains under the 15% P5 warning threshold β no aggregate P5 finding raised. Scheduled cancellation rate (0/1 = 0%) remains well under the 30% P6 warning threshold β no aggregate P6 finding raised.
π€ Generated by DevOps Health Check agentic workflow Β· Run #31237438735 Β· 2026-08-08T03:34:23 UTC
Generated by DevOps Daily Health Check Β· auto Β· 158.6 AIC Β· β 9.46 AIC Β· β 20.3K Β· β·
π₯ Daily Health Check β 2026-08-08
Status: π΄ 2 critical Β· π‘ 2 warnings Β· π΅ 0 info
Since yesterday: π 2 new Β· β 1 resolved Β· π 2 unchanged
π New Findings (2)
π΄ Evaluation workflow failed on main: "Select available Copilot token from pool" step
pipeline:evaluation:evaluate-/-vally-(dotnet-template-engine):select-available-copilot-token-from-pool:failuremain, started 2026-08-08T00:19:38Z) failed in job evaluate / vally (dotnet-template-engine) at the Select available Copilot token from pool step. The run took an unusually long 180.85 minutes before failing/completing (vs. a typical ~20β60 min), suggesting the token-pool selection logic stalled or exhausted retries rather than failing fast. This directly correlates with the eval-duration critical finding below β the same run is the primary driver of today's inflated 14-day average.π΄ Evaluation average duration exceeds 55-minute critical threshold
resource:eval-duration:criticalevaluation.ymlrun duration over the last 14 days (main branch, 24 completed runs) is 57.7 minutes, above the 55-minute critical threshold (and well above the 50-minute warning threshold). This is skewed by the 180.85-minute outlier run above; excluding it, most other runs in the sample ranged ~18β70 minutes, several already exceeding the 50-minute warning band.π Investigation Results
β Resolved Since Yesterday (1)
Evaluation workflow run cancelled (deploy-dashboard job)pipeline:evaluation:deploy-dashboard:timeoutmainβ no cancelled runs occurred. This appears to have been an isolated single occurrence, as previously assessed.π Existing Findings (2)
π‘ Warning β Orphan plugin: dotnet-experimental not in marketplace.json Β· first seen 2026-05-14 Β· 62 occurrences
Fingerprint:
infra:orphan-plugin:dotnet-experimentalCategory: Infrastructure Β· Severity: π‘ Warning
The plugin directory
plugins/dotnet-experimental/has a validplugin.jsonbut is not listed in.github/plugin/marketplace.json. Consumers cannot discover this plugin. 86 days outstanding.Links: plugin.json Β· marketplace.json
Suggested action: Either add
dotnet-experimentaltomarketplace.jsonif it is ready for consumers, or remove the plugin directory if it is no longer needed.π‘ Warning β Validate PAT Pool workflow failed: Build summary step Β· first seen 2026-08-05 Β· 3 occurrences
Fingerprint:
pipeline:validate-pat-pool:validate-copilot-pat-pool:build-summary:failureCategory: Pipeline Β· Severity: π‘ Warning
The
Validate PAT Poolworkflow'sValidate Copilot PAT Pooljob failed again at the Build summary step on today's scheduled run (Run #31236108858), the third consecutive occurrence. All 10 individualValidate COPILOT_PAT_Nsteps succeeded β only the summary-generation step fails, continuing to look like a scripting/formatting bug rather than an actual PAT pool health problem.Links: Run #31236108858
Suggested action: Investigate the summary-generation logic in
validate-pat-pool.yml's "Build summary" step; three consecutive failures strongly suggest a persistent bug rather than a transient issue β worth prioritizing a fix.π Trends (7-day)
π€ Generated by DevOps Health Check agentic workflow Β· Run #31237438735 Β· 2026-08-08T03:34:23 UTC