Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

trying out adding benchmarks to github actions CI #524

Merged
merged 10 commits into from
Oct 31, 2021

Conversation

ajfriend
Copy link
Contributor

@ajfriend ajfriend commented Oct 9, 2021

As worried about in #523.

My GH Actions knowledge is rusty, so more than happy if anyone has suggestions to clean up the test-bench.yml.

@coveralls
Copy link

coveralls commented Oct 9, 2021

Coverage Status

Coverage decreased (-0.2%) to 98.513% when pulling ffde47e on ajfriend:gh_actions_benchmarks into f2786a4 on uber:master.

@ajfriend ajfriend marked this pull request as ready for review October 9, 2021 05:43
Copy link
Collaborator

@nrabinowitz nrabinowitz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I can see the benchmark output in the checks. Would be awesome if we had some coveralls-style way of reporting this, but 🤷

.github/workflows/test-bench.yml Outdated Show resolved Hide resolved
Copy link
Collaborator

@isaacbrodsky isaacbrodsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +34 to +37
- name: Build
run: |
cd build
make
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now we don't have a way to indicate building just the benchmarks without running them in the same invocation. If the build part needs to be sped up we could turn off some other build options (building tests, etc.) in CMake, or add a "build without run" option for benchmarks.

@ajfriend
Copy link
Contributor Author

I wonder if there's some way we can collect and monitor benchmark performance, like https://github.com/marketplace/actions/continuous-benchmark or https://werat.dev/blog/running-benchmarks-for-pull-requests-via-github-actions/?

That'd be nice! I'm deferring it to #530 just so I can get this landed, finally 😄

@ajfriend ajfriend merged commit 69bbe0f into uber:master Oct 31, 2021
@ajfriend ajfriend deleted the gh_actions_benchmarks branch October 31, 2021 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants