Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for benchmark test from forks #1345

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

YigitElma
Copy link
Collaborator

@YigitElma YigitElma commented Nov 4, 2024

Benchmark action requires write access for commenting which forks don't have. This change makes the benchmark workflow coming from a fork run on our CI and thus have proper access.

This shouldn't change the general permissions for fork repos, so someone in the DESC team must still approve to test to run for such PRs.

https://github.com/thollander/actions-comment-pull-request?tab=readme-ov-file#permissions

  • Make sure this is secure! Check from a fork repo!

Copy link
Contributor

github-actions bot commented Nov 4, 2024

|             benchmark_name             |         dt(%)          |         dt(s)          |        t_new(s)        |        t_old(s)        | 
| -------------------------------------- | ---------------------- | ---------------------- | ---------------------- | ---------------------- |
 test_build_transform_fft_midres         |     -4.19 +/- 5.76     | -2.66e-02 +/- 3.66e-02 |  6.08e-01 +/- 2.4e-02  |  6.35e-01 +/- 2.8e-02  |
 test_build_transform_fft_highres        |     -0.19 +/- 2.33     | -1.86e-03 +/- 2.33e-02 |  1.00e+00 +/- 1.2e-02  |  1.00e+00 +/- 2.0e-02  |
 test_equilibrium_init_lowres            |     +2.14 +/- 5.81     | +8.23e-02 +/- 2.23e-01 |  3.93e+00 +/- 1.6e-01  |  3.85e+00 +/- 1.5e-01  |
 test_objective_compile_atf              |     +2.59 +/- 5.59     | +2.05e-01 +/- 4.44e-01 |  8.14e+00 +/- 3.1e-01  |  7.94e+00 +/- 3.1e-01  |
 test_objective_compute_atf              |     +3.27 +/- 4.18     | +3.52e-04 +/- 4.50e-04 |  1.11e-02 +/- 3.3e-04  |  1.08e-02 +/- 3.0e-04  |
 test_objective_jac_atf                  |     +2.46 +/- 3.23     | +4.76e-02 +/- 6.26e-02 |  1.99e+00 +/- 4.7e-02  |  1.94e+00 +/- 4.2e-02  |
 test_perturb_1                          |     +5.91 +/- 2.82     | +7.59e-01 +/- 3.62e-01 |  1.36e+01 +/- 1.4e-01  |  1.28e+01 +/- 3.3e-01  |
 test_proximal_jac_atf                   |     +1.30 +/- 1.40     | +1.07e-01 +/- 1.16e-01 |  8.33e+00 +/- 6.7e-02  |  8.23e+00 +/- 9.4e-02  |
 test_proximal_freeb_compute             |     -0.98 +/- 0.94     | -1.80e-03 +/- 1.72e-03 |  1.82e-01 +/- 1.5e-03  |  1.84e-01 +/- 8.6e-04  |
 test_build_transform_fft_lowres         |     +0.47 +/- 2.15     | +2.42e-03 +/- 1.10e-02 |  5.14e-01 +/- 6.9e-03  |  5.12e-01 +/- 8.5e-03  |
 test_equilibrium_init_medres            |     +0.52 +/- 0.74     | +2.10e-02 +/- 2.97e-02 |  4.05e+00 +/- 2.2e-02  |  4.03e+00 +/- 1.9e-02  |
 test_equilibrium_init_highres           |     +0.36 +/- 0.63     | +1.92e-02 +/- 3.40e-02 |  5.38e+00 +/- 2.9e-02  |  5.36e+00 +/- 1.8e-02  |
 test_objective_compile_dshape_current   |     +0.13 +/- 0.97     | +4.98e-03 +/- 3.68e-02 |  3.81e+00 +/- 2.8e-02  |  3.80e+00 +/- 2.4e-02  |
 test_objective_compute_dshape_current   |     +1.56 +/- 1.70     | +5.57e-05 +/- 6.06e-05 |  3.63e-03 +/- 5.2e-05  |  3.58e-03 +/- 3.2e-05  |
 test_objective_jac_dshape_current       |     +2.76 +/- 6.35     | +1.09e-03 +/- 2.51e-03 |  4.07e-02 +/- 2.0e-03  |  3.96e-02 +/- 1.5e-03  |
 test_perturb_2                          |     +0.13 +/- 0.66     | +2.24e-02 +/- 1.13e-01 |  1.73e+01 +/- 5.2e-02  |  1.73e+01 +/- 1.0e-01  |
 test_proximal_freeb_jac                 |     +0.68 +/- 1.67     | +4.96e-02 +/- 1.22e-01 |  7.38e+00 +/- 8.2e-02  |  7.33e+00 +/- 9.1e-02  |
 test_solve_fixed_iter                   |     -0.25 +/- 54.25    | -1.24e-02 +/- 2.70e+00 |  4.96e+00 +/- 1.9e+00  |  4.98e+00 +/- 1.9e+00  |

@YigitElma YigitElma self-assigned this Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
override codecov Override codecov
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant