Skip to content

Modify shared benchmarking ops to allow optional failure when comparing commits#500

Merged
stephenworsley merged 7 commits intoSciTools:mainfrom
trexfeathers:benchmark-fail
Apr 15, 2025
Merged

Modify shared benchmarking ops to allow optional failure when comparing commits#500
stephenworsley merged 7 commits intoSciTools:mainfrom
trexfeathers:benchmark-fail

Conversation

@trexfeathers
Copy link
Copy Markdown
Contributor

I believe we've been missing all regressions since #361.

scitools-ci[bot]
scitools-ci bot previously requested changes Apr 14, 2025
Copy link
Copy Markdown
Contributor

@scitools-ci scitools-ci bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@trexfeathers trexfeathers marked this pull request as draft April 14, 2025 14:59
@trexfeathers
Copy link
Copy Markdown
Contributor Author

trexfeathers commented Apr 15, 2025

What the test commits show

commit description GHA what this demonstrates
deddb11 The proposed change 40513851523 The proposed change does NOT cause CI failures when no performance regressions are found
ecf8ee9 introducing a false regression 40516320872 The proposed change will cause a CI failure when a performance regression is found
7f1a3fe removing the proposed code change 405746246131 Without the proposed change, the CI will pass even when regressions are found - i.e. in the current status quo we are not noticing performance regressions e.g. when Dask changes

Footnotes

  1. This is actually the GHA run from 993f9d9 - I had to resolve conflicts with main before anything would run

@trexfeathers trexfeathers marked this pull request as ready for review April 15, 2025 12:56
Copy link
Copy Markdown
Contributor

@stephenworsley stephenworsley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Definitely worth getting this in ASAP.

@stephenworsley stephenworsley merged commit 3913ff3 into SciTools:main Apr 15, 2025
16 checks passed
trexfeathers added a commit to trexfeathers/SciTools.github that referenced this pull request Apr 15, 2025
pp-mo added a commit to SciTools/.github that referenced this pull request Apr 24, 2025
* Add suite of benchmarking templates.

* Include benchmark templates in config file.

* Fix config file.

* Add templates for benchmark GHA workflows.

* Include benchmark workflows in config file.

* Incorporate changes from SciTools/iris-esmf-regrid#500.

* Remove uses of REPONAME placeholder text.

* Fix omissions in _templating_include.json.

* Remove Iris references from README.

* Clarify the use of files with ASV plugins.

* Correct _asv_delegated_abc references.

---------

Co-authored-by: Patrick Peglar <patrick.peglar@metoffice.gov.uk>
@scitools-ci scitools-ci bot removed this from 🚴 Peloton May 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants