Skip to content

ci(mergify): checks timeout at 2×p95 for the release queue (rail G11; batching blocked by plan tier) - #16066

Merged
diegosouzapw merged 1 commit into
release/v3.8.52from
ci/rail-g11-mergify-queue-tuning
Oct 10, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.52from
ci/rail-g11-mergify-queue-tuning

Conversation

@diegosouzapw

Copy link
Copy Markdown
Owner

Summary

Rail G11 (v3.8.54) asks the merge queue for batching ~5 and a timeout ≈ 2×p95 of CI. This PR lands the timeout half. The batching half is blocked by the Mergify plan tier: on this repo, batch_size/batch_max_wait_time already got every PR dequeued with "⚠ The 'Merge Queue Batch' feature requires a higher subscription tier" (live finding on #7205, removed in #7220). Re-adding batch_size: 5 would break the queue, so it stays out until the owner decides to upgrade the subscription.

What changed

  • .mergify.yml → queue_rules.release.checks_timeout: 240 min. The YAML comment records the measurement. merge_conditions, queue_conditions and the governance header are untouched.

  • p95 is measured over the last 100 successful quality.yml runs (2026-09-28 → 2026-10-08), wall-clock createdAt → updatedAt, so runner queueing counts:

    mean p50 p90 p95 max
    34.5 min 1.8 min 101.0 min 119.75 min 468.6 min

    2 × 119.75 = 239.5, rounded up to 240 min. The distribution is bimodal: path-filtered docs-only runs take about 2 min, and full runs have a median of about 72 min.

  • Before this change the queue used Mergify's default checks_timeout: auto. Per the docs, auto applies no timeout until the queue has enough CI history on the branch. Our queue is serial and lightly used, so a hung check could stall it with no limit.

  • docs/ops/MERGE_TRAIN.md: the runbook said the queue "batches up to 10 PRs and bisects". The queue has been serial since fix(ci): drop Mergify batch settings (paid-tier feature — queue failed with 'Cannot use Merge Queue batch') #7220, so the runbook now says so.

  • tests/unit/mergify-queue-config.test.ts (new contract test) pins:

    • the release queue conditions (label=queue, -draft, -conflict, base~=^release/) and the auto-merge label;
    • fail-closed merge conditions (#check-pending=0, #check-success>=1, the Merge integrity anchor);
    • checks_timeout ≥ 2 × the measured p95, kept as a commented constant;
    • absence of the paid-tier batch keys and of deprecated keys (allow_inplace_checks, autoqueue, speculative_checks).

Validation

Gate Command Result
YAML parse npx --yes js-yaml .mergify.yml OK
Mergify validator curl -F data=@.mergify.yml https://api.mergify.com/v1/validate/ 403 (needs auth); Mergify's own config check on this PR is the authoritative validation
Contract test node --import tsx/esm --test tests/unit/mergify-queue-config.test.ts 6/6 pass
Mutation check checks_timeout: 200 min → re-run 1 fail (guard works), reverted
ESLint npx eslint tests/unit/mergify-queue-config.test.ts clean
Prettier npx prettier --check (3 files) clean
Docs npm run check:docs-all EXIT=0 (links + fabricated-docs PASS)
Pre-commit husky (lint-staged, docs-sync, any-budget, tracked-artifacts, ai-attribution) pass

Open risk: the Mergify docs do not say whether checks_timeout is tier-gated; they say so for batching only. If Mergify's config check on this PR flags it, drop the key and fall back to auto.

⚠️ base-red inherited: #15306

… queue (rail G11)

Rail G11 (v3.8.54) asks for a batched queue (~5) with a timeout of about 2x the
CI p95. Only the timeout half can ship: batching is a paid Mergify tier on this
repo (live dequeue "Cannot use Merge Queue batch", #7220), so batch_size stays
out and the contract test guards against re-adding it.

- queue_rules.release: checks_timeout 240 min = 2 x the 119.75 min p95 wall-clock
  of the last 100 successful quality.yml runs (2026-09-28..10-08).
- docs/ops/MERGE_TRAIN.md: the runbook claimed the queue batches up to 10 PRs
  and bisects; it has been serial since #7220.
- tests/unit/mergify-queue-config.test.ts: pins queue/merge conditions, the
  timeout floor (2 x measured p95) and the absence of paid/deprecated keys.
@mergify

mergify Bot commented Oct 9, 2026

Copy link
Copy Markdown

Merge Protections

🔴 1 of 2 protections blocking · waiting on 🙋 you

Protection Waiting on
🔴 🚦 Auto-queue 🙋 you
🟢 📃 Configuration Change Requirements —

🔴 🚦 Auto-queue

Waiting for

  • label = queue
This rule is failing.

When all merge protections are satisfied and these conditions match, this pull request will be queued automatically.

  • label = queue

Show 1 satisfied protection

🟢 📃 Configuration Change Requirements

Mergify configuration change

  • any of:
    • check-success = @mergify/Configuration changed
    • check-success = @mergify/Configuration has been deleted

@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

CI Coverage Report

  • Coverage job: skipped
  • PR test policy: success

Coverage artifact was not available for this run.

@diegosouzapw
diegosouzapw merged commit 62228d3 into release/v3.8.52 Oct 10, 2026
31 of 52 checks passed
diegosouzapw added a commit that referenced this pull request Oct 10, 2026
Brings in #16073 (RELEASE_STRATEGY.md, ROADMAP See-also), #16180
(dev:candidate, RELEASE_CHECKLIST, meta.json), #16172 (dist-tag.mjs and the
dormant LTS workflows) and #16066 (Mergify checks_timeout). package.json keeps
both release:dry-run-lts-cut and labels:sync.

Now that the base carries them, the LTS-cut dry-run uses the real
scripts/release/dist-tag.mjs resolver (the built-in fallback is gone), treats a
dormant workflow missing at the cut source as NOT READY, and the runbook /
GO-NO-GO docs link RELEASE_STRATEGY.md and name the real forward-port secret
and nightly publish variable. The G11 fixture is the real .mergify.yml.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant