Report what every CI job spends of its retry budget (GH-3787)

[jeremydmiller]

Closes #3787.

CIAzureServiceBus reported green for four consecutive main runs while burning 22 of its
25-retry budget on every one of them. That was 85% of all flakiness in the repository, and nothing
in the GitHub UI distinguishes a job at 22/25 retries from one at 0/25.

The annotations were not the safety net they looked like

Bobcat logs every retry, and Nuke's GitHub Actions sink does turn those Serilog warnings into real
::warning annotations — so the Annotations panel was never empty. It is worse than empty: GitHub
caps warning annotations at ten per check run, first-wins. On run 30759855844, the ASB job that
spent 22 retries:

$ gh api ".../actions/jobs/91528323381/logs" | grep -c FLAKY
66                       # 22 flaky tests, logged three ways

$ gh api ".../check-runs/91528323381/annotations?per_page=100" --paginate -q length
10

Ten [FLAKY] lines. You would see ten at 10 retries too, and ten at 50. The count is the exact
quantity GitHub discards, and the one line that does carry it (239 passed (0 retries, ...)) lives
in the log of a job that passed, which nobody has a reason to open.

What this adds

output	where	why
Per-project retry table	$GITHUB_STEP_SUMMARY	On the run page without opening a log, and without competing for the ten annotation slots
One ::warning with the count	Annotations panel	Emitted before the per-test [FLAKY] warnings, so it is inside the ten that survive
aggregate.json per run	artifact	The baseline the next run diffs against

The third is the one that matters. The absolute 22 was never the signal — the step from 1 to 22
between two adjacent runs was, and no baseline existed anywhere to notice it against. The new
Flakiness roll-up job aggregates all thirty jobs and reports the delta:

## Flakiness roll-up

**6 retries** across 30 reporting job(s), 8214 tests.
Baseline — last `main` run #30761362744: **24**, -18.

| job | retries | baseline | Δ | passed on retry | failed | indeterminate |
|---|--:|--:|:--|--:|--:|--:|
| CIMQTT             | 3 |  0 | **+3** | 3 | 1 | 0 |
| CISqlite           | 2 |  2 | ·      | 2 | 0 | 0 |
| CIAzureServiceBus  | 1 | 22 | -21    | 1 | 0 | 0 |

It also names any job that reported no ledger at all. Bobcat prints its summary only at the end,
so a job killed by the 20-minute cap reports nothing — not even a partial count — and that silence
would otherwise read as clean.

Deliberately not a gate

An earlier suggestion of "fail past a threshold of 5 retries" is too loose: a suite legitimately
sitting at 3 today would be one bad day from a red main. A nonzero count is something a human
explains — a review norm, not a cliff. Nothing here can fail a build; flakiness-report.sh exits 0
unconditionally.

Verification

• Ran CISqlite locally under GITHUB_ACTIONS=true with two tests rigged (marker file) to fail on
  first attempt and pass in the retry process. All three outputs produced the real numbers:
  step-summary table with **2** bolded, the counted ::warning ahead of the [FLAKY] lines, and
  the JSON ledger naming both tests.
• Exercised flakiness-report.sh against synthesized ledgers for the no-baseline, with-baseline,
  all-clean, and empty-input cases. A FLAKINESS_BASELINE_FILE escape hatch exists for exactly this.
• The delta path was confirmed by feeding it a baseline of 24 and a run of 6.

The real test is this PR's own Flakiness roll-up job — it is the first run, so it will report
"no baseline found" and absolute numbers only.

🤖 Generated with Claude Code

https://claude.ai/code/session_0116vfBcKwcjWn8msM4ZjkuA