ci: stop running the micro-benchmark comparison on every PR

[justinchuby]

The Benchmarks workflow posted a 🔴 Benchmark Regression Detected comment on PRs that had changed nothing relevant. The cause is not the comparison logic — it is the machine.

A GitHub-hosted runner is not a quiet machine. It is shared, its neighbours are invisible to us, and we cannot verify it was idle for the window we timed.

Why this is worse than publishing nothing. A red flag that fires on noise trains readers to ignore it. Once the header stops carrying information, a real regression posted the same way gets dismissed too — so the failure mode is not "a useless comment", it is "a gate nobody believes".

Why the base-vs-PR design did not rescue it. Timing base first and PR second in the same job cancels some runner-to-runner variation, but it cannot cancel within-job contention: the two halves still run at different moments against different neighbours. The thresholds were calibrated against ~27% worst-case measured noise, and regressions worth catching are often smaller than that — so the signal sat below the noise floor, not above it.

Nothing is lost. This workflow never gated anything; its own header says Does NOT block CI. The real regression gates are the throughput floors and dispatch-reachability tests in crates/onnx-genai-bench/tests/profile_native.rs, which run on real hardware and are untouched here.

What changed

• Removed the pull_request trigger; the job is now workflow_dispatch-only, taking a PR number as input.
• A manual run checks out the default branch rather than the PR, so it now resolves the head sha and base ref via gh pr view and fetches the head explicitly.
• Updated the remaining github.event.pull_request.* references (concurrency group, comment lookup, comment post) to use the input.

The workflow was also disabled in the Actions UI (gh workflow disable) so the noise stopped immediately rather than waiting on this merge.

Verified

• python -c "import yaml; yaml.safe_load(...)" → parses; triggers are exactly ['workflow_dispatch'].
• No remaining github.event.pull_request references in the file.
• benchmark.yml is the only workflow that runs cargo bench / criterion — grep across .github/workflows/*.yml confirms no other PR-triggered benchmark job exists.

Not verified: the manual workflow_dispatch path has not been executed end-to-end (it needs a real dispatch on a macOS runner). The PR-metadata resolution is a straightforward substitution, but it is untested — worth a trial dispatch before anyone relies on it.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.64%. Comparing base (4a9f4ec) to head (a014194).
⚠️ Report is 69 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1490      +/-   ##
==========================================
+ Coverage   82.10%   82.64%   +0.54%     
==========================================
  Files          12       12              
  Lines        5471     5475       +4     
  Branches     5471     5475       +4     
==========================================
+ Hits         4492     4525      +33     
+ Misses        780      757      -23     
+ Partials      199      193       -6     

Flag	Coverage Δ
cli-ort-linux	82.60% <ø> (?)
cli-ort-windows	82.19% <ø> (+0.09%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 3 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.