Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete Cartis Roberts benchmark set #435

Merged
merged 30 commits into from
Mar 21, 2023
Merged

Complete Cartis Roberts benchmark set #435

merged 30 commits into from
Mar 21, 2023

Conversation

segsell
Copy link
Member

@segsell segsell commented Feb 9, 2023

Complete the Medium-Scale Test Problem Set by Cartis & Roberts.

@codecov
Copy link

codecov bot commented Mar 6, 2023

Codecov Report

Merging #435 (d82778d) into main (bea7abb) will decrease coverage by 0.11%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #435      +/-   ##
==========================================
- Coverage   93.05%   92.94%   -0.11%     
==========================================
  Files         250      249       -1     
  Lines       18597    18311     -286     
==========================================
- Hits        17305    17019     -286     
  Misses       1292     1292              
Impacted Files Coverage Δ
...c/estimagic/benchmarking/get_benchmark_problems.py 93.00% <ø> (-0.10%) ⬇️
src/estimagic/benchmarking/more_wild.py 100.00% <ø> (ø)

... and 1 file with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@segsell segsell requested a review from janosg March 7, 2023 10:01
@timmens
Copy link
Member

timmens commented Mar 16, 2023

A few notes:

  1. Q: Do we still need this warning in get_benchmark_problems.py?
    def _get_raw_problems(name):
     if name == "more_wild":
         raw_problems = MORE_WILD_PROBLEMS
     elif name == "cartis_roberts":
         warnings.warn(
             "Only a subset of the cartis_roberts benchmark suite is currently "
             "implemented. Do not use this for any published work."
         )
         raw_problems = CARTIS_ROBERTS_PROBLEMS
  2. Q: Do we still need this line in pyproject.toml?
    [tool.pytest.ini_options]
    filterwarnings = [
    # ...
    "ignore:Only a subset of the cartis_roberts",
    # ...
    ]
  3. Note: Benchmark runtime with scipy_lbgfsb:

@segsell segsell removed the request for review from janosg March 20, 2023 15:06
@segsell segsell requested a review from timmens March 20, 2023 16:22
Copy link
Member

@timmens timmens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a fantastic PR 🎉 . Thank you a lot!

Note: In the future, we will try to capture code-coverage of jitted functions. I have opened an issue accordingly: #447

@timmens timmens merged commit 3ac4dfb into main Mar 21, 2023
@timmens timmens deleted the complete_cartis_roberts branch March 21, 2023 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants