CI: auto-refresh benchmarks README table + charts after main merges#23
Merged
CI: auto-refresh benchmarks README table + charts after main merges#23
Conversation
…er main merges benchmarks/README.md has an auto-generated table between <!--BENCHMARK_TABLE_START/END--> markers plus two SVG charts (chart.svg, chart-delta.svg). These are produced by 'npm run bench:report' but nothing refreshed them automatically — the committed numbers drifted from reality as main received encode/decode changes between manual regenerations. This workflow runs on push-to-main (path-filtered to sources that can affect bench numbers — excludes README.md and SVGs so the refresh PR itself does not retrigger the loop), reruns bench:report with median-of-5 and CPU 0 pinned, and opens a sticky chore PR on branch chore/bench-refresh when the output differs. The PR carries labels chore + benchmarks and is force-updated on each subsequent main merge instead of stacking duplicate PRs. Main continues to be modified only by reviewed commits — the workflow has contents: write solely to push the chore/bench-refresh branch; PRs against main still go through normal review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
.github/workflows/bench-refresh.yamlthat runs on every push tomain(path-filtered to sources that can affect bench numbers), rerunsnpm run bench:report -w @bufbuild/protobuf-benchmarkswith median-of-5 and CPU 0 pinning, and opens a sticky chore PR on branchchore/bench-refreshwhen the output differs from what's committed.Why
benchmarks/README.mdhas an auto-generated table between<!--BENCHMARK_TABLE_START/END-->markers plus two SVG charts (chart.svg,chart-delta.svg). These are produced bybench:reportbut nothing refreshed them automatically — the committed numbers drifted as main received encode/decode changes between manual regenerations. #22 just shipped with a README table captured mid-authoring, not post-merge.Design choices
contents: writeonly to pushchore/bench-refresh. Refreshes still go through normal review — reviewer sees the delta and decides.packages/protobuf/**,benchmarks/src/**,benchmarks/scripts/**,benchmarks/proto/**,benchmarks/package.json. The refresh PR only touchesbenchmarks/README.md+ two*.svgs, none of which are in the list, so merging it does not loop.taskset -c 0. Same noise controls asbench-matrixandrun-matrix-ci.sh—BENCH_REPORT_RUNS=5is already thereport.tsdefault since Drop toBinaryFast from main, archive L1+L2, add upstream baseline #22, workflow pins explicitly as documentation.peter-evans/create-pull-request@v7. Standard action for this pattern; force-updates the branch on subsequent main merges instead of stacking duplicate PRs.chore+benchmarks— both created on the repo before opening this PR.Test plan
mainthat touches bench-influencing code should trigger the workflow.chore/bench-refresh.workflow_dispatchcan be used to force-regenerate on demand.