ci: Releasing benchmarks and benchmarking PR#2422
Conversation
sozercan
changed the title
JaydipGabani
force-pushed
the
master
branch
5 times, most recently
from
ec42591 to
635fe99
Compare
Codecov ReportBase: 53.71% // Head: 53.61% // Decreases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## master #2422 +/- ##
==========================================
- Coverage 53.71% 53.61% -0.10%
==========================================
Files 117 117
Lines 10281 10281
==========================================
- Hits 5522 5512 -10
- Misses 4341 4347 +6
- Partials 418 422 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
|
Benchmark job result is located here |
| - name: Download release benchmark file | ||
| uses: robinraju/release-downloader@v1.6 | ||
| id: get-latest-benchmark | ||
| continue-on-error: true |
There was a problem hiding this comment.
We are not really executing rest of the steps if the file is missing.
If we dont continue on error, the whole job fails and I think it be best if we not fail on missing previous benchmark file, just skip all the steps of the workflow if release benchmark is missing
| id: get-latest-benchmark | ||
| continue-on-error: true | ||
| with: | ||
| latest: true |
There was a problem hiding this comment.
is this going to take latest semver or latest release?
There was a problem hiding this comment.
This is going to take latest tagged release
There was a problem hiding this comment.
I couldn't find an action that returns the latest semver. Still looking, I think we could sort it also after retrieving every release if this is needed.
There was a problem hiding this comment.
can we use the github api to get recent releases then find the latest semver?
There was a problem hiding this comment.
Yip! It took some time, but I was able to get the latest release semver version by calling API and sorting
JaydipGabani
force-pushed
the
master
branch
3 times, most recently
from
4a6a9ba to
55ea1bf
Compare
| export GOPATH="$HOME/go" | ||
| PATH="$GOPATH/bin:$PATH" | ||
| go install golang.org/x/perf/cmd/benchstat@latest | ||
| benchstat release_benchmarks.txt pr_benchmarks.txt > benchstat.txt |
There was a problem hiding this comment.
what happens if release_benchmarks.txt doesn't exist?
There was a problem hiding this comment.
If the release_benchmark.txt doesn't exist steps.get-latest-benchmark.outcome results in failure instead of success and hence we skip the rest of the steps.
| with: | ||
| issue-number: ${{ github.event.pull_request.number }} | ||
| body: | | ||
| Here is how this PR compares againt latest released version in terms of benchmark. Unless this PR containts huge updates, if the performace is significantly lower consider optimizing your changes to improve the performace closest to latest released benchmark. |
There was a problem hiding this comment.
| Here is how this PR compares againt latest released version in terms of benchmark. Unless this PR containts huge updates, if the performace is significantly lower consider optimizing your changes to improve the performace closest to latest released benchmark. | |
| This PR compares its performance to the latest released version. If it performs significantly lower, consider optimizing your changes to improve the performance. |
Signed-off-by: Sertac Ozercan <sozercan@gmail.com>
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
JaydipGabani
force-pushed
the
master
branch
7 times, most recently
from
2e6118a to
c0ccd31
Compare
JaydipGabani
force-pushed
the
master
branch
17 times, most recently
from
788ff01 to
6073a59
Compare
JaydipGabani
force-pushed
the
master
branch
8 times, most recently
from
b3dde7e to
a9c3998
Compare
Signed-off-by: Jaydip Gabani <gabanijaydip@gmail.com>
3 participants
What this PR does / why we need it:
Releasing benchmark stats with each release and comparing benchmark data from each PR against latest released benchmark stats.
Which issue(s) this PR fixes (optional, using
fixes #<issue number>(, fixes #<issue_number>, ...)format, will close the issue(s) when the PR gets merged):Fixes #692
Special notes for your reviewer: