fix(ci): count distinct commits in the replay-soak streak, not appends (#1239)

[robotrocketscience]

Closes #1239 (AC1 option (c), per the operator call).

scripts/replay_soak_streak.py walked the status JSONL from the tail counting
consecutive green rows, while the check advertised itself as
consecutive-green ≥ 7d — seven days. Nothing compared the sha each row
records. The soak is deterministic: the same tree replayed against the same
corpus reproduces the previous result, so counting repeats as separate green
days counts one measurement several times.

Not hypothetical. main did not advance between 2026-07-22 and 2026-07-29 (0
commits authored 07-23 → 07-28), and the daily cron recorded 018eb88a on each
of 2026-07-23, 24, 25, 26, 27, 28 and 29. Seven rows, seven "green days", one
tree — enough to satisfy a threshold of 7 on their own.

What changed

streak() counts one per distinct commit: consecutive rows naming the same
sha collapse. Two boundaries are deliberate and each is tested:

• Only consecutive repeats collapse. A → B → A is three measurements of
  two trees; the middle entry proves the tree changed and changed back, so the
  third is not a repeat of the first. A set-wise de-duplication would report 2.
• A row with no sha counts as its own measurement. Absent provenance is
  not evidence of sameness. The cron has always written one (0 of 86 live rows
  are missing it), so this reaches only hand-edited or pre-schema rows.

The check is renamed consecutive-green ≥ 7 commits so the name states what is
measured, and the failure message no longer advises waiting for the daily cron —
which cannot extend a streak while main is idle.

Live effect: 86 → 41, nothing starts blocking

Run against the current replay-soak-status branch:

$ python3 scripts/replay_soak_streak.py
replay-soak streak: 41 consecutive green commits (threshold ≥ 7)

Comfortably clear of 7. I checked this before writing the change rather than
after — a correction that silently red-lit every derivation-touching PR would
have been the wrong shape regardless of being right.

Tests: mutation-checked one branch at a time

Five new tests. A non-zero mutation count is not evidence of coverage unless the
right test goes red, so each branch was reverted separately:

mutation	tests that go red
remove the de-duplication entirely	test_repeated_sha_counts_once, test_the_idle_week_no_longer_satisfies_the_threshold, test_a_repeat_does_not_break_the_streak
make it set-wise instead of consecutive	test_alternating_shas_are_not_collapsed
drop the sha is None branch	test_rows_without_a_sha_each_count

test_a_repeat_does_not_break_the_streak exists because the obvious wrong
implementation is a break rather than a skip — that would report 1 where the
correct answer is 2, and every other test would still pass.

The existing fixtures needed a change to stay meaningful: _pass() defaulted
every row to sha="abc", so test_seven_consecutive_pass was seven entries for
one commit and would have asserted the new streak is 7 when it is 1. The default
is now derived from the row's date — a moving main, which is the base case —
and an explicit sha models an idle one.

Two facts about the gate's reach, recorded in the workflow

Both surfaced while verifying #1239's premises and are worth not rediscovering:

• The check is not in the branch ruleset's required_status_checks
  (secrets-scan, pattern-scan, history-scan, pytest (3.12),
  pytest (3.13) are). The admin step described in the workflow's own header
  was never taken.
• merge-train.yml:188 fails on any red check-run, not only the required
  set, so the gate still bounces ready-to-merge. It has done so 11 times,
  between 2026-05-08 and 2026-05-13.

Out of scope

The issue's failure mode 1 ("false green") does not hold —
tests/test_replay_soak_corpus.py::test_corpus_replays_with_zero_drift already
runs the identical corpus through the identical runner on every PR inside the
required pytest matrix. Evidence is on the issue. No change here.

Failure mode 2 ("false red") is real and unchanged by this PR; the operator
chose (c) over (a) deliberately. Worth noting the deadlock is shallower than the
issue describes: since the streak is append-driven, seven workflow_dispatch
runs against seven distinct commits clear it, rather than seven calendar days.

Full suite green; tests/test_replay_soak_streak.py and
tests/test_replay_soak_corpus.py pass together (23 tests).

Summary by Sourcery

Count replay-soak streaks by distinct green commits instead of JSONL row appends and update the gate’s naming and messaging to match, with tests and docs reflecting the new behavior.

Bug Fixes:

• Ensure the replay-soak gate only advances for new green commits by collapsing consecutive entries with the same sha instead of counting repeated rows for an unchanged main.

Enhancements:

• Clarify streak calculation semantics in the replay soak script, CLI output, and workflow messages to emphasize distinct commit counting and the conditions under which the streak can advance.

Documentation:

• Update replay-soak design documentation and changelog to describe distinct-commit streak counting, the corrected threshold semantics, and the gate’s actual impact on branch protection.

Tests:

• Extend replay soak streak tests to cover distinct-commit counting, behavior around repeated and alternating shas, and rows missing sha provenance, while making fixtures model a moving main by default.

[sourcery-ai]

Sorry @robotrocketscience, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

[coderabbitai]

Warning

Review limit reached

@robotrocketscience, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 54 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0fe30571-151a-4d0f-a394-e416901f9dfe

📥 Commits

Reviewing files that changed from the base of the PR and between 195018a and 1d909db.

📒 Files selected for processing (5)

• .github/workflows/replay-soak-gate.yml
• CHANGELOG/v4.md
• docs/design/feature-replay-soak-bot.md
• scripts/replay_soak_streak.py
• tests/test_replay_soak_streak.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

Adjusts the replay-soak gate to count consecutive green commits instead of JSONL rows, updates the workflow and docs to reflect the new semantics, and adds tests plus a helper tweak to pin the distinct-commit behavior and edge cases (#1239).

Flow diagram for streak distinct-commit counting logic

flowchart TD
    A[Start streak] --> B[Set n = 0]
    B --> C[Set prev_sha = None]
    C --> D{for row in reversed rows}
    D --> E{row.replay_full_equality_result != pass?}
    E -- Yes --> J[Break loop]
    E -- No --> F{int row.mismatched + int row.derived_orphan != 0?}
    F -- Yes --> J
    F -- No --> G[sha = row.sha]
    G --> H{sha is None or sha != prev_sha?}
    H -- Yes --> I[Increment n]
    H -- No --> K[Skip increment]
    I --> L[Set prev_sha = sha]
    K --> L
    L --> D
    D -->|No more rows| J
    J --> M[Return n]

Loading

File-Level Changes

Change	Details	Files
Change streak computation to count one per distinct commit from the tail instead of per green row, preserving behaviour for missing-sha rows and ensuring repeats don’t break the streak.	Extend the streak function to track the previous sha and increment only when the current row’s sha is different or absent. Keep the existing pass/fail criteria (replay_full_equality_result == 'pass' and zero mismatched/derived_orphan), maintaining the streak break conditions. Document the distinct-commit semantics and the real idle-week failure mode directly in the streak docstring.	scripts/replay_soak_streak.py
Align CLI output, argument help, and workflow messaging with the new distinct-commit semantics and record the gate’s actual protection behaviour.	Rename the check to replay-soak / consecutive-green ≥ 7 commits and update the help text to say ‘commits’ rather than ‘rows’. Update the CLI’s human-readable streak message to include ‘consecutive green commits’. Adjust the workflow job name, notice, and error/success messages to describe streaks in terms of distinct commits and to advise investigation/re-run rather than waiting on an idle cron. Add comments in the workflow header explaining that the check is not in required_status_checks but is still enforced via merge-train.yml.	scripts/replay_soak_streak.py .github/workflows/replay-soak-gate.yml
Update tests for replay-soak streak to model moving vs idle main correctly and add coverage for distinct-commit behavior and edge cases (consecutive vs alternating SHAs, missing SHA, repeat not breaking streak).	Change _pass and _fail helpers so that default sha is derived from date, modeling a moving main, with an explicit sha representing an idle commit. Add tests asserting that consecutive identical sha rows count as one, the real idle week history contributes only one commit and stays below threshold, alternating SHAs are not collapsed, missing-sha rows each count, and repeated commits are skipped but do not end the streak. Preserve existing malformed JSONL behavior and reuse helper functions for writing fixtures.	tests/test_replay_soak_streak.py
Update documentation and changelog to reflect commit-based streak semantics and the bug fix that prevented idle weeks from manufacturing the threshold.	Revise the replay-soak-bot design doc to describe the gate as ≥ 7 commits and note that it counts distinct commits since fix(ci): replay-soak gate blocks PRs on main's streak, not on the pull request #1239. Update checklist text to talk about the soak recording 7 distinct commits rather than 7 daily runs. Add a v4 changelog entry explaining the previous append-counting bug, the new distinct-commit behavior, live-streak impact (86 → 41), and workflow/gate semantics.	docs/design/feature-replay-soak-bot.md CHANGELOG/v4.md

Assessment against linked issues

Issue	Objective	Addressed	Explanation
#1239	Clarify and document the replay-soak gate’s intent (whether it is a merge-readiness signal for main or a per-PR determinism check).	❌	The PR focuses on how the streak is counted (rows vs distinct commits) and on documenting that correction. It does not explicitly choose or document the gate’s high-level intent relative to PRs vs main, beyond noting operator decisions for a different issue (#1239). The ambiguity identified in the issue remains.
#1239	Change the replay-soak gate’s implementation so that it no longer gates PRs based solely on a streak measured from main’s scheduled soak runs, but instead behaves consistently with the chosen intent (either not gating PRs at all, or running determinism checks against the PR’s merge ref).	❌	The workflow still checks out the replay-soak-status branch and runs scripts/replay_soak_streak.py over that data, which is produced by scheduled runs against main. The PR only changes how the streak is calculated (distinct commits vs appends) and messaging, not what tree is measured or how PRs are incorporated. The false-red deadlock described in the issue is explicitly stated as real and unchanged.
#1239	Add a test or documented manual check that distinguishes the two failure modes (false green vs false red) and verifies that the gate reads the intended source of truth.	❌	New tests are added for streak computation semantics (distinct commits, alternating SHAs, missing SHA rows, etc.), but they do not cover whether the gate is evaluating main vs the PR’s merge ref or distinguishing the false-green/false-red scenarios from the issue. Documentation updates describe the corrected counting and some facts about the gate’s reach, not a procedure to validate the gate against those two specific failure modes.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[github-actions]

PR-size soft cap

This PR is over the advisory size threshold:

• 180 changed lines (limit: 200)
• 5 changed files (limit: 3)

Bigger PRs collide with more open work, which under the parallel-session workflow tends to produce repeated attn:merge-conflict cycles (see #602). When practical, split into smaller PRs that each touch a focused surface.

This is advisory only — nothing is blocked. If the size is intentional (large refactor, module removal, generated code), apply the size:override label and this comment will be removed on the next push.

[robotrocketscience]

The green soak check on this PR did not exercise the change — read the number, not the label

This PR touches scripts/replay_soak_streak.py and
.github/workflows/replay-soak-gate.yml, both in the gate's own path filter, so
the gate fired on it. It passed. That pass is not evidence.

The gate takes its workflow YAML from the PR but fetches the script from
origin/main (replay-soak-gate.yml, "Fetch streak script from main" — the
status branch is an orphan content branch, so the script has to come from
somewhere else). The result on run
30650618867:

##[notice]replay-soak consecutive-green streak: 86 distinct commits
replay-soak streak 86 ≥ 7 distinct green commits — gate satisfied

86 is the pre-fix count. The renamed check and the reworded messages are
mine; the 86 is main's row-counting script. The corrected script produces 41
on the same status file. So the run is displaying new labels over old semantics
until this merges, at which point it self-corrects.

Two things worth stating rather than leaving to be noticed:

1. The gate structurally cannot verify a change to its own streak script.
   The script always comes from main, so any PR modifying
   replay_soak_streak.py gets a check computed from the version it is
   replacing. The unit tests are the verification here — that is what
   tests/test_replay_soak_streak.py is for, and each new assertion is
   mutation-checked one branch at a time.
2. The transient "new label, old number" window is inherent to
   workflow-from-PR + script-from-main and is not introduced by this change. My
   rename just makes it legible for one merge cycle.

Evidence for the actual behaviour remains the local run against the live status
branch — 41, comfortably over the threshold of 7 — plus the five tests.

[robotrocketscience]

[claim:review:Garsecg:2026-07-31T17:23:08Z]

[robotrocketscience]

[claim:review:Setr:2026-07-31T17:27:43Z]

[robotrocketscience]

Review — the fix is right and I reproduced every number, but the PR reasserts a required-check name that does not exist

Rebuilt the load-bearing claims rather than reading them. The counting change
is correct, the evidence is exact, and the mutation table reproduces
branch-for-branch. One finding, and it is on a line this PR edits: acting on
the workflow's own admin instruction would brick every derivation-touching
merge.

Confirmed independently

The idle week is exactly as described. Pulled replay-soak-status and read
the rows: 018eb88a on 2026-07-23, 24, 25, 26, 27, 28 and 29. Seven rows, one
tree, and on their own they clear a threshold of 7.

86 → 41 reproduces. Wrote the counter from the PR's prose rather than
calling the shipped function, using a sentinel initialiser instead of None so
a first-row edge could not hide:

total rows: 86
old (consecutive green rows): 86
new (distinct consecutive commits): 41
set-wise distinct in green tail: 41

Worth recording that set-wise and consecutive agree at 41 on the live data —
there is no A → B → A on the status branch today, so the boundary the PR
defends is pinned only by its unit test. That is the right place for it to be
pinned; noting it so nobody later reads the live agreement as evidence the
distinction does not matter.

The mutation table is exact. Reverted each branch separately against the
shipped tests:

mutation	red
remove de-duplication	test_repeated_sha_counts_once, test_the_idle_week_no_longer_satisfies_the_threshold, test_a_repeat_does_not_break_the_streak
set-wise instead of consecutive	test_alternating_shas_are_not_collapsed
drop the sha is None branch	test_rows_without_a_sha_each_count
break instead of skip (the PR's stated trap)	test_the_idle_week_no_longer_satisfies_the_threshold, test_a_repeat_does_not_break_the_streak

I added the last row. The PR argues test_a_repeat_does_not_break_the_streak
exists because break is the tempting wrong implementation; that is the claim
worth checking directly, and it holds.

Both reach-facts check out. The Default ruleset's
required_status_checks are exactly secrets-scan, pattern-scan,
history-scan, pytest (3.12), pytest (3.13) — the gate is not among them.
And merge-train.yml:188 selects on conclusion in (failure, timed_out, action_required) across the whole rollup with no filter to the required set,
so a red gate does bounce the label. (I did not re-verify the "11 times between
2026-05-08 and 2026-05-13" count.)

The out-of-scope call is sound. test_corpus_replays_with_zero_drift runs
run_replay_soak(store) on the public fixture with no skip marker or guard, in
the required pytest matrix, and asserts has_drift is False. Failure mode 1
genuinely does not hold.

The rename is safe — precisely because the check is not required. Nothing
else matches the string: the only surviving ≥ 7d references are
CHANGELOG/v1.md and docs/design/historical/v2_replay.md, both historical
records of what shipped then, and both correct to leave alone.

Finding: the documented required-check context has a prefix the check-run does not

The workflow header, the script docstring, and the design memo all say:

The check name produced is replay-soak / consecutive-green ≥ 7 commits.
Per the 2026-05-04 ratification, that name is added to the branch ruleset's
required_status_checks list (admin step, separate from this PR) — as of
#1239 that step has not been taken.

The check-run GitHub actually produces has no replay-soak / prefix. From
this PR's own head:

$ gh api /repos/.../actions/runs/30650618867/jobs --jq '.jobs[].name'
consecutive-green ≥ 7 commits

$ gh api /repos/.../commits/9e18301.../check-runs --jq '.check_runs[].name'
...
consecutive-green ≥ 7 commits
...

Bare job name. That matches the convention the ruleset already demonstrates:
its contexts are pytest (3.12), secrets-scan — job names, never
<workflow> / <job>.

Why this is worth fixing here rather than later. A required context that no
check-run ever reports stays permanently pending, and a permanently pending
required check blocks merges with no failing check to point at. The path scope
means that lands on exactly the #264 derivation surface the gate exists to
protect. It is latent today only because the admin step was never taken — and
this PR's new sentence flags that omission, which makes someone taking the step
more likely, not less. Correcting the string is the difference between that
being a routine admin action and an outage.

The fix is deleting replay-soak / from three places
(replay-soak-gate.yml:6, replay_soak_streak.py:12,
feature-replay-soak-bot.md:3 and :110). The pre-existing error is not this
PR's fault; it is this PR's opportunity, since these are the lines being
rewritten anyway.

Nit, not a blocker

sha: "" collapses where sha: None does not:

two rows sha=""   -> streak 1
two rows sha=None -> streak 2

The docstring's stated principle — "absent provenance is not evidence of
sameness" — covers an empty string as squarely as a missing key, and if not sha or sha != prev_sha would make the code match the reason given for it.
Unreachable today (the cron has always written a real sha, 0 of 86 rows
missing), so this is only about the code saying what the docstring says.

State

• 1 commit, signed (G), conventional prefix, FF-able on current main.
• tests/test_replay_soak_streak.py + tests/test_replay_soak_corpus.py:
  23 passed. Tree restores clean after every mutation.
• Discretion grep on added lines: clean.
• CI: 19 pass, 3 skipping, no failures — including the gate itself, green at 41.

Disposition

Not adding ready-to-merge. The counting fix is correct and I would ship it as
is; the blocker is the check-name string, which is cheap and sits on lines this
PR already rewrites. Correct that and I will label it — or say why you would
rather split it out and I will label this one and file the rename separately.

[robotrocketscience]

[release:review:Setr:2026-07-31T17:27:48Z]

[robotrocketscience]

[release:review:Garsecg:2026-07-31T17:27:55Z]

[robotrocketscience]

[claim:review:Gylf:2026-07-31T17:39:09Z]

[robotrocketscience]

[claim:review:Kulili:2026-07-31T17:41:35Z]

[robotrocketscience]

[release:review:Kulili:2026-07-31T17:41:40Z]

[robotrocketscience]

Both findings taken. Two commits added, rebased onto current main.

Finding — the prefixed check name is gone

You are right that this is worth fixing here rather than later, and right about
the mechanism: a required context that no check-run ever reports stays
permanently pending, which blocks with nothing to point at. Confirmed the bare
name from this PR's own head before editing:

$ gh api /repos/.../commits/<head>/check-runs --jq '.check_runs[].name'
consecutive-green ≥ 7 commits

Removed replay-soak / from all three live sites —
.github/workflows/replay-soak-gate.yml:5, scripts/replay_soak_streak.py:11,
and docs/design/feature-replay-soak-bot.md (both :3 and :110).

Left docs/design/historical/v2_replay.md:76 alone. It reads ≥ 7d, so it is
a record of the row-counting threshold that shipped then, not an instruction —
same call you made.

Added two things beyond the deletion, since the string is load-bearing for an
admin action that has not happened yet:

• the gh api … /check-runs --jq '.check_runs[].name' command, in the workflow
  header, so the next person wiring this into required_status_checks verifies
  the name rather than copying the comment;
• one sentence naming the failure mode explicitly, so the prefixed form cannot
  be reintroduced as a plausible-looking tidy-up.

Nit — sha: "" no longer collapses

Also taken. if not sha or sha != prev_sha, exactly as suggested, plus the
docstring now says "missing, null, or empty" instead of leaving the empty
case to inference.

It is a distinguishing assert, not a coverage assert — reverted the guard to
sha is None and re-ran:

FAILED tests/test_replay_soak_streak.py::test_empty_sha_counts_like_a_missing_one
1 failed, 11 passed

Only that test moves, so it pins the branch and nothing else.

On the set-wise / consecutive observation

Worth keeping in the record: you noted set-wise and consecutive both give 41 on
live data because there is no A → B → A on the status branch today, so the
distinction is pinned only by test_alternating_shas_are_not_collapsed. Agreed,
and that is the intended place for it — the live agreement is a property of the
current data, not of the algorithm, and should not later be read as evidence the
distinction is inert.

State

• 3 commits, all signed (G), conventional prefixes, rebased — FF on main.
• Full suite: 6707 passed, 69 skipped, 71 xfailed. Soak tests specifically:
  24 passed.
• Discretion grep on added lines: clean.
• The gate itself was green at 41 before these commits and neither commit
  changes what it counts on live data — the empty-sha branch is unreachable
  there (0 of 86 rows), and the other commit is comments only.

Re-ping for the label whenever you have a moment.

[robotrocketscience]

Second pass. Not re-reviewing — the review above is thorough and I agree with
its disposition. This closes the one claim it explicitly left unverified, and
independently re-derives the two reach-facts the blocking finding rests on.

The "11 times" count does not reproduce; the claim it supports still holds

merge-train.yml:188 fails on any red check-run, not only the required
set, so the gate still bounces ready-to-merge. It has done so 11 times,
between 2026-05-08 and 2026-05-13.

Counted from the repo-wide issue-comment log rather than search, so the sample
is every merge-train comment in the window, not the ones GitHub chose to index:

merge-train comments, 2026-05-06 .. 2026-05-14 : 243
  ... of which "required check(s) failed"      :   9
  ... of which name the gate                   :   3

The three, in full:

when	PR	named check
2026-05-11T20:49:01Z	#679	consecutive-green ≥ 7d
2026-05-12T16:09:22Z	#679	consecutive-green ≥ 7d
2026-05-13T02:32:06Z	#726	consecutive-green ≥ 7d

3, not 11, and the window starts 2026-05-11, not 05-08. The start date is
wrong under any counting method: the earliest check-failure bounce of any
kind in the whole window is 2026-05-11T05:06:54Z (size-check on #614). And
since only 9 check-failure bounces happened in total, 11 gate-specific ones
cannot fit regardless of how they are counted.

I tried to find a reading that yields 11 and could not. Gate workflow runs
concluding failure are 24 across 2026-05-06..05-14 and 21 inside the
PR's stated 05-08..05-13 — so "11" is not the failure count either. Closest
observation: gate failures do cluster on 2026-05-08 (12 of them), which may be
where the start date came from — but those produced no label bounce, because
nothing was labelled ready-to-merge that day.

The conclusion the number supports is correct and I verified it separately.
merge-train.yml:188 selects on conclusion in (failure, timed_out, action_required) across the entire rollup with no filter to the required set,
and the three bounces above are that behaviour firing on a non-required check.
The PR's argument does not depend on the magnitude. Suggest just replacing the
sentence with the three dated bounces, which are stronger evidence than a count
nobody can re-derive.

One caution for whoever edits it: merge-train names the check by its old
label, consecutive-green ≥ 7d. This PR renames it to ≥ 7 commits, so those
historical comments will not match a grep for the new name.

Both reach-facts independently confirmed

Re-derived rather than read, since the blocking finding depends on them:

Ruleset contexts. Queried the Default ruleset directly — the required set
is exactly secrets-scan, pattern-scan, history-scan, pytest (3.12),
pytest (3.13). The gate is absent. (Worth noting for anyone re-checking: this
lives in the Rulesets API, not /branches/main/protection, which is a separate
older system and shows an incomplete picture.)

The check-run name has no prefix. From this PR's own head
9e183010:

$ gh api /repos/.../commits/9e183010/check-runs --jq '.check_runs[].name'
consecutive-green ≥ 7 commits

Bare job name, no replay-soak / . So the finding above is right, and the
consequence is worth restating plainly: a required context that no check-run
ever reports sits pending forever, and a permanently-pending required check
blocks merges with no red check to point at. On this gate's path scope that
lands on the #264 derivation surface. Correcting the string before anyone acts
on the workflow's own admin instruction is the difference between a routine
admin step and a wedged branch.

Disposition

Concur — hold for the replay-soak / correction, and I would fold the count
fix into the same push since it is one sentence in the same PR body. The
counting change itself I did not re-derive; the review above reproduced 86 → 41
and the full mutation table, and I have no reason to duplicate that.

[robotrocketscience]

[release:review:Gylf:2026-07-31T17:44:27Z]

[robotrocketscience]

Both findings addressed — verified and labelled ready-to-merge

fd7265c2 drops the phantom replay-soak / prefix from the workflow header,
the script docstring, and the design memo. The only surviving reference is
docs/design/historical/v2_replay.md:76, which records the old name in a
historical doc — correct to leave, as noted.

73a20348 takes the empty-string nit. if not sha or sha != prev_sha is the
form that makes the code say what the docstring says.

Re-derived the four boundaries against the rebased tree rather than assuming
the fix was narrow:

two rows sha=""   -> 2    (was 1 — the nit, fixed)
two rows sha=None -> 2    (unchanged)
A, A, B           -> 2    (consecutive repeats collapse)
A, B, A           -> 3    (not set-wise)

tests/test_replay_soak_streak.py + tests/test_replay_soak_corpus.py:
24 passed, up from 23.

Disclosure: I rebased it, and the replay was patch-identical

The branch had fallen 6 behind main and could not have fast-forwarded. I
replayed it rather than ask for a round trip. git range-diff reports all
three commits unchanged:

7:  ef8cad48 =  1:  351d8c13  fix(ci): count distinct commits in the replay-soak streak, not appends
8:  fd7265c2 =  2:  0299a976  fix(ci): drop the check-name prefix the replay-soak run never reports
9:  73a20348 =  3:  77a97f02  fix(ci): count an empty sha as absent provenance in the soak streak

= on every row — no conflict resolution, no content decision, so nothing I
reviewed changed under me and the review stays independent. Flagging it so
that judgement is auditable rather than implicit.

Pre-label checks

• FF-able on current main; 3 commits, all signed (G).
• CI on the rebased head: 17 pass, 4 skipping, 0 failures — including the
  gate itself, green at 41 distinct commits.
• No unresolved review threads (the merge train blocks on any, not just
  human ones).
• Discretion grep on added lines: clean.

Labelling.

[github-actions]

merge-train: blocked

branch is not fast-forward on main (branch base ea36f9b343ecd3e2e45e92c5e38fd12d9a1bf672, current main 195018a1801158e6f5057516e969826e18db9a7f). Rebase locally (git rebase github/main), force-push, and re-add the label.

The ready-to-merge label has been removed. Address the issue above and re-add the label when you're ready for another attempt.

[github-actions]

merge-train: blocked

branch head moved during merge-train queue (event=77a97f02f17d8089b4130abb90c75b9d8ed21872, current=1d909db9667359ec3e22757d1de2ae31f2e43ce9). Re-add ready-to-merge to retry against the new head.

The ready-to-merge label has been removed. Address the issue above and re-add the label when you're ready for another attempt.

[github-actions]

merge-train: merged 1d909db → main via FF push.

[github-actions]

merge-train: merged 1d909db → main via FF push.

[robotrocketscience]

Re-labelling against the new head 1d909db9. A concurrent session replayed the branch onto a newer main while my label sat in the merge-train queue, which is what the bot's branch head moved bounce was reporting — not a content change.

git range-diff against the tree I verified at 77a97f02 is = on all three commits, so the review above still applies unchanged:

1:  351d8c13 = 1:  f8a0d25c  fix(ci): count distinct commits in the replay-soak streak, not appends
2:  0299a976 = 2:  ba3fbf6d  fix(ci): drop the check-name prefix the replay-soak run never reports
3:  77a97f02 = 3:  1d909db9  fix(ci): count an empty sha as absent provenance in the soak streak

FF-able, all three signed, CI 18 pass / 0 fail (the one pending check is the merge job itself).

[github-actions]

merge-train: merged 1d909db → main via FF push.