ci(perf): PR-level BDN regression gate with delta comment (#192) - #272
Merged
Conversation
Closes #192. New pr-benchmarks.yaml: runs BenchmarkDotNet on the PR's HEAD and on its base's tip, diffs mean-time and allocations per benchmark, posts an upsert-style delta comment on the PR (found by marker string `<!-- pr-benchmarks-report -->`, edited in place on push), and fails the workflow when any regression exceeds the threshold AND the PR does not carry the `perf-impact-acknowledged` label. Complements benchmarks.yaml (post-merge trend chart to gh-pages). That workflow is backward-looking; this is forward-looking, catching regressions before merge. Design: - Trigger: pull_request against main/vNext, paths src/, benchmarks/, workflow itself. - Thresholds via env: time > +20%, allocations > +50% (adjust in the workflow env block; comment table always shows all deltas). - Comment upsert via marker string using `gh api` — no third-party action dependency for the comment step. - Base BDN run is `|| true`-guarded so a PR that ADDS benchmarks where the base branch has none surfaces as "new benchmark" rather than a hard fail. - Concurrency group per PR with cancel-in-progress: a mid-run push cancels the older invocation so the final comment reflects HEAD, not a stale interim commit. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
PR benchmarks vs base (6b7444b)Thresholds: time > +20% or allocations > +50% trigger a gate failure (label
|
This was referenced Jul 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #192.
Summary
New
.github/workflows/pr-benchmarks.yaml: runs BDN on PR HEAD + PR base, diffs mean-time + allocations per benchmark, posts an upserted delta comment (marker:<!-- pr-benchmarks-report -->), fails the PR when any regression exceeds threshold ANDperf-impact-acknowledgedlabel is absent.Thresholds (env-configurable)
TIME_REGRESSION_PCT: 20ALLOC_REGRESSION_PCT: 50Comment table always shows all deltas; only regressions past the threshold trigger the gate.
Design points
gh api(find-by-marker → PATCH; else POST). No third-party action dependency for the comment step.cancel-in-progress— mid-run pushes abort the stale run so the final comment reflects HEAD.|| true-guarded so a PR that ADDS benchmarks where base has none surfaces as "new benchmark" rather than hard fail.Stacked-PR base
Base =
vNext. Tenth of the 11-PR batch (all workflow additions; #259 investigation next).Protected-file guard
New workflow → protected. Expected admin-bypass at merge.
Test plan
src/**orbenchmarks/**onvNextruns the workflowperf-impact-acknowledgedlabel overrides the gate on a regressing PR