Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/workflows/actions-queue-health.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: GitHub Actions queue health

on:
schedule:
- cron: "7 * * * *"

concurrency:
group: github-actions-queue-health
cancel-in-progress: false

permissions:
contents: read
actions: read
pull-requests: read

jobs:
collect:
name: Collect exact-head queue evidence
runs-on: ubuntu-24.04
permissions:
contents: read
actions: read
pull-requests: read
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0
with:
egress-policy: audit

- name: Checkout trusted queue-health source
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
persist-credentials: false

- name: Collect read-only repository and runner evidence
env:
GH_TOKEN: ${{ secrets.PR_REVIEW_MERGE_TOKEN || secrets.OPENCODE_APPROVE_TOKEN }}
run: |
if [ -z "${GH_TOKEN:-}" ]; then
echo "::error::PR_REVIEW_MERGE_TOKEN or OPENCODE_APPROVE_TOKEN is required for cross-repository queue reads."
exit 1
fi
echo "::add-mask::$GH_TOKEN"
python3 scripts/ci/actions_queue_health.py \
--allowlist config/actions_queue_health_repositories.json \
--output-json "$RUNNER_TEMP/actions-queue-health.json" \
--output-html "$RUNNER_TEMP/actions-queue-health.html"

- name: Upload queue-health evidence
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: github-actions-queue-health-${{ github.run_id }}
path: |
${{ runner.temp }}/actions-queue-health.json
${{ runner.temp }}/actions-queue-health.html
if-no-files-found: error
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,11 @@ Semantic Versioning where the repository publishes a release.

### Added

- Added a read-only hourly GitHub Actions queue-health report for the central
repository, TEPP, naruon, and contextual-orchestrator, with exact pull-request
head classification, runner-assignment evidence, bounded queue-age SLO
warnings, deterministic JSON, accessible HTML, and no cancellation or merge
mutation.
- Added an hourly organization commercial-readiness coordinator that discovers writable repositories, honors enabled dedicated writer leases and fully paginated live writer runs, refetches exact repository/workflow/run/PR state before dispatch, rotates bounded review-repair and opt-in NVIDIA OpenCode product-development targets, fails nonzero on fleet-wide inspection or dispatch outages, retains three-day JSON receipts, and keeps the existing 15-minute merge scheduler authoritative.
- Added a dedicated OriginWeave hourly caller that invokes the product-neutral central scheduler with the exact repository, protected `main` branch, one-dispatch budget, two-hour same-head retry floor, non-cancelling single-flight heartbeat, job-scoped OIDC, and only the established scheduler credentials.
- Added a trusted pull-request comment router for `@cwl-noema-review` and review-only `@opencode-agent` dispatches, with an organization sweep, exact-head receipts, repository allowlisting, fixed runners, immutable checkout pins, and a permanent 100% statement/branch/docstring quality gate.
Expand All @@ -32,6 +37,9 @@ Semantic Versioning where the repository publishes a release.

### Fixed

- Validate collector-produced canonical pull-request identity at report time and
retry one transient incomplete raw response before failing queue-health collection
with repository context; exact-head classification remains fail-closed.
- Parsed `opencode.jsonc` as JSONC (stripping `//` and `/* */` comments outside string literals) in the reasoning-effort guard and its contract tests, instead of raw `json.loads`, which rejected the file the moment it carried its first explanatory comment (added for the `contextual-orchestrator` provider block) with `Expecting property name enclosed in double quotes`. Comment markers inside string values, such as the `$schema` URL, are left untouched.
- Download the pinned `uv` 0.12.1 exporter from the official GitHub Releases URL instead of `releases.astral.sh`, which now returns HTTP 403 and blocks org-wide OpenCode `coverage-evidence`. The SHA-256 pin is unchanged. The opener may follow one hop onto `release-assets.githubusercontent.com` or `objects.githubusercontent.com` and still rejects every other host, userinfo, non-HTTPS scheme, and nondefault port (ContextualWisdomLab/.github#1109).
- Compared the trusted `uv` executable's post-install `--version` output against the real GitHub Releases build's full string, `uv 0.12.1 (x86_64-unknown-linux-gnu)`, instead of the bare `uv 0.12.1` the prior check required; the genuine release binary always prints the target triple, so every installation was failing the pin check immediately after the archive download itself was fixed (ContextualWisdomLab/.github#1109).
Expand Down
8 changes: 8 additions & 0 deletions config/actions_queue_health_repositories.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"repositories": [
"ContextualWisdomLab/.github",
"ContextualWisdomLab/TEPP",
"ContextualWisdomLab/contextual-orchestrator",
"ContextualWisdomLab/naruon"
]
}
40 changes: 40 additions & 0 deletions docs/doctoring/actions-queue-health.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# GitHub Actions queue-health evidence

The scheduled `actions-queue-health.yml` workflow reads a fixed allowlist of
CWL repositories once per hour and publishes a JSON report plus a keyboard-
readable HTML report as an artifact. The collector uses only `gh api` reads
through the configured cross-repository `PR_REVIEW_MERGE_TOKEN` or
`OPENCODE_APPROVE_TOKEN`; it fails visibly when neither credential is present.
It does not cancel runs, mutate branches, dispatch workflows, or alter merge
gates, and it never relies on the central repository's scoped `GITHUB_TOKEN`
for sibling-repository reads.

The report schema is `actions.queue_health.v1`. Each observed run records its
repository, pull-request number, head SHA, event, run attempt, concurrency
group (or an explicit unavailable marker), queue age, job state, and runner
assignment. A run is `current_head` only when its linked open pull request and
head SHA match. Stale linked runs are `obsolete`; runs without a pull-request
link are `unlinked`. Queued evidence remains incomplete even when a report is
successfully produced.

Queued runs use run-level evidence because GitHub has not assigned their jobs;
only current-head `in_progress` runs make the additional jobs API read needed
to inspect a concrete runner assignment.

List endpoints use GitHub CLI pagination with at most 20 pages of 100 records.
An incomplete, malformed, or larger response fails closed instead of silently
claiming that the visible page is the whole queue.

The default queue-age SLO is 900 seconds. A current-head job that remains
unassigned beyond that limit produces a warning and an explicit manual action
to inspect runner capacity, billing, runner-group policy, environment
approval, and concurrency saturation. The workflow intentionally remains
read-only and fail-closed when GitHub API or runner evidence is unavailable.
Paged API reads are not atomic; changing totals are retained only when the
collected records cover the largest observed total, and the report remains
explicitly an observation rather than a merge decision.

The allowlist is deliberately explicit in
`config/actions_queue_health_repositories.json`; adding a repository requires
review of its governance and data boundary. This first slice does not claim
that a queued run is obsolete or safe to cancel.
Loading
Loading