-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BENCH: sync codspeed-benchmarks with BLAS-benchmarks #4763
Conversation
Expanding the benchmark suite of course takes longer. Especially given that codspeed runs things under callgrind, it is not entirely unexpected to be slow. I'm not sure what are their free usage limits, Previously: the codspeed run was under 5 minutes (e.g. https://github.com/OpenMathLib/OpenBLAS/actions/runs/9632137639/job/26564891418) I wonder if we should keep the a small set of system sizes and run s-/d-/c-/z- variants? |
CodSpeed Performance ReportMerging #4763 will not alter performanceComparing Summary
Benchmarks breakdown
|
Yes I don't think it makes sense to expand the codspeed benchmarks to the same granularity and complexity as the standalone benchmark suite - apart from (potential direct monetary) cost, having a CI job run for hours can be a major nuisance when waiting to commit something. Small set of sizes and either of the two complex types in addition to the existing double precision real is probably good enough, and we are checking one architecture only anyway |
Okay, down to 35 mins for the benchmarks themselves + some 15 mins build time with |
Sync the benchmark suite we run on codspeed with http://www.openmathlib.org/BLAS-Benchmarks/#summarylist?sort=0&dir=asc: