perf(daemon): add sync bench comparator

[andreabadesso]

Motivation

Second piece of the sync-benchmarking infrastructure (follows #396). Given two bench-results-*.json files — one for master, one for a PR branch — we need a rigorous way to decide whether a change actually moved the needle. A single point-estimate comparison is misleading at the run counts this harness produces; this PR bootstraps a 95% CI on the median delta for each metric instead.

Stacked on #396 — targets feat/daemon-sync-bench-harness. Once #396 lands, rebase this onto master.

Acceptance Criteria

• yarn bench:compare --baseline a.json --candidate b.json reads two bench outputs and emits a markdown comparison to stdout
• Bootstrap resampling (default 10,000 samples, deterministic via fixed seed) produces a 95% CI on median delta for each metric
• Each metric is flagged 🟢 (CI fully below 0), 🔴 (CI fully above 0), ⚪ (CI crosses 0 — noise), or ⚠️ (missing/invalid)
• Tool exits 0 regardless of regressions — output is informational, suitable for piping to gh pr comment
• bench-sync.ts output now includes raw samples: number[] per metric (required for bootstrap)

Self-test (same code, two runs, 5 measured runs each)

Scenario: VOIDED_TOKEN_AUTHORITY (66 events)

• 19 metrics compared
• 18 ⚪ noise (CI crosses 0 — as expected for identical code)
• 1 🔴 false positive on markUtxosAsVoided (+20.9%, CI [+2.0%, +37.8%])

One false positive across 19 independent 95% CIs is exactly what theory predicts (≈0.95 expected). This validates the decision to keep the report informational. The totalMs CI of [-14.5%, +19.5%] quantifies the other known limitation: scenarios need to grow to the thousands-of-events range before sub-20% deltas are reliably detectable.

Example output

## Sync benchmark comparison

**Scenario:** `VOIDED_TOKEN_AUTHORITY` (66 events)
**Runs:** baseline=5, candidate=5, warmup=1/1
**Bootstrap samples:** 10000, seed: 42
**Verdict:** 🟢 0 improvements · 🔴 1 regression · ⚪ 18 noise · ⚠️ 0 skipped

| metric | baseline p50 (ms) | candidate p50 (ms) | Δ | 95% CI | |
|---|---:|---:|---:|---|:---:|
| totalMs | 1253.6 | 1194.6 | -4.7% | [-14.5%, +19.5%] | ⚪ |
| handleVertexAccepted | 490.6 | 438.1 | -10.7% | [-22.7%, +19.8%] | ⚪ |
| markUtxosAsVoided | 2.369 | 2.865 | +20.9% | [+2.0%, +37.8%] | 🔴 |
...

Implementation notes

• Deterministic bootstrap (mulberry32 PRNG, fixed seed) so the same inputs always produce the same CIs — comparable across runs
• Median rather than mean — less sensitive to the inevitable slow outlier run
• Scenario mismatch between files aborts with exit 1
• Missing spans on either side are reported as ⚠️ rather than silently dropped

Checklist

• If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged
• Make sure either the unit tests and/or the QA tests are capable of testing the new features (self-test on identical code documented above)
• Make sure you do not include new dependencies in the project unless strictly necessary and do not include dev-dependencies as production ones. (no new deps)

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0ce302c8-4bf8-4601-badf-725a9ceed6ed

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/daemon-bench-comparator

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}