Skip to content

CI: auto-refresh benchmarks README table + charts after main merges#23

Merged
intech merged 1 commit intomainfrom
ci/bench-refresh-on-main-merge
Apr 21, 2026
Merged

CI: auto-refresh benchmarks README table + charts after main merges#23
intech merged 1 commit intomainfrom
ci/bench-refresh-on-main-merge

Conversation

@intech
Copy link
Copy Markdown

@intech intech commented Apr 21, 2026

Summary

Adds .github/workflows/bench-refresh.yaml that runs on every push to main (path-filtered to sources that can affect bench numbers), reruns npm run bench:report -w @bufbuild/protobuf-benchmarks with median-of-5 and CPU 0 pinning, and opens a sticky chore PR on branch chore/bench-refresh when the output differs from what's committed.

Why

benchmarks/README.md has an auto-generated table between <!--BENCHMARK_TABLE_START/END--> markers plus two SVG charts (chart.svg, chart-delta.svg). These are produced by bench:report but nothing refreshed them automatically — the committed numbers drifted as main received encode/decode changes between manual regenerations. #22 just shipped with a README table captured mid-authoring, not post-merge.

Design choices

  • Sticky chore PR, not direct push to main. Main is protected; the workflow has contents: write only to push chore/bench-refresh. Refreshes still go through normal review — reviewer sees the delta and decides.
  • Self-retrigger guard via path filter. Trigger includes packages/protobuf/**, benchmarks/src/**, benchmarks/scripts/**, benchmarks/proto/**, benchmarks/package.json. The refresh PR only touches benchmarks/README.md + two *.svgs, none of which are in the list, so merging it does not loop.
  • Median-of-5 + taskset -c 0. Same noise controls as bench-matrix and run-matrix-ci.shBENCH_REPORT_RUNS=5 is already the report.ts default since Drop toBinaryFast from main, archive L1+L2, add upstream baseline #22, workflow pins explicitly as documentation.
  • peter-evans/create-pull-request@v7. Standard action for this pattern; force-updates the branch on subsequent main merges instead of stacking duplicate PRs.
  • Labels chore + benchmarks — both created on the repo before opening this PR.

Test plan

  • Merge this PR.
  • Next push to main that touches bench-influencing code should trigger the workflow.
  • If the regenerated table/SVGs differ, a PR appears on branch chore/bench-refresh.
  • Merging that refresh PR does not retrigger the workflow (path filter excludes README/SVG).
  • workflow_dispatch can be used to force-regenerate on demand.

…er main merges

benchmarks/README.md has an auto-generated table between
<!--BENCHMARK_TABLE_START/END--> markers plus two SVG charts
(chart.svg, chart-delta.svg). These are produced by
'npm run bench:report' but nothing refreshed them automatically — the
committed numbers drifted from reality as main received encode/decode
changes between manual regenerations.

This workflow runs on push-to-main (path-filtered to sources that can
affect bench numbers — excludes README.md and SVGs so the refresh PR
itself does not retrigger the loop), reruns bench:report with
median-of-5 and CPU 0 pinned, and opens a sticky chore PR on branch
chore/bench-refresh when the output differs. The PR carries labels
chore + benchmarks and is force-updated on each subsequent main merge
instead of stacking duplicate PRs.

Main continues to be modified only by reviewed commits — the workflow
has contents: write solely to push the chore/bench-refresh branch; PRs
against main still go through normal review.
@intech intech merged commit 7193f24 into main Apr 21, 2026
27 of 28 checks passed
@intech intech deleted the ci/bench-refresh-on-main-merge branch April 21, 2026 10:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant