Skip to content

Conversation

@MegaRedHand
Copy link
Contributor

Description

This PR adds option flags (proof mode and trace/memory dumping) to the benchmarks script, to mimic the CI benches.

@MegaRedHand MegaRedHand added the pipelines This PR/issue is exclusively about improving our CI label Jun 9, 2023
@codecov
Copy link

codecov bot commented Jun 9, 2023

Codecov Report

Merging #1219 (57a34b7) into main (73539d3) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1219   +/-   ##
=======================================
  Coverage   97.59%   97.59%           
=======================================
  Files          89       89           
  Lines       36163    36163           
=======================================
  Hits        35293    35293           
  Misses        870      870           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@github-actions
Copy link

github-actions bot commented Jun 9, 2023

Benchmark Results for unmodified programs 🚀

Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 12.143 ± 0.061 12.061 12.228 1.00 ± 0.01
head blake2s_integration_benchmark 12.098 ± 0.041 12.059 12.177 1.00
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 4.225 ± 0.059 4.175 4.333 1.00
head compare_arrays_200000 4.235 ± 0.065 4.180 4.373 1.00 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 2.699 ± 0.012 2.687 2.722 1.00 ± 0.01
head dict_integration_benchmark 2.698 ± 0.011 2.684 2.718 1.00
Command Mean [s] Min [s] Max [s] Relative
base factorial_multirun 4.350 ± 0.022 4.330 4.401 1.00 ± 0.01
head factorial_multirun 4.341 ± 0.031 4.315 4.426 1.00
Command Mean [s] Min [s] Max [s] Relative
base fibonacci_1000_multirun 3.748 ± 0.075 3.681 3.951 1.01 ± 0.02
head fibonacci_1000_multirun 3.729 ± 0.020 3.709 3.778 1.00
Command Mean [ms] Min [ms] Max [ms] Relative
base field_arithmetic_get_square_benchmark 170.9 ± 3.4 167.3 177.3 1.01 ± 0.02
head field_arithmetic_get_square_benchmark 169.2 ± 1.2 167.6 171.2 1.00
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 11.004 ± 0.163 10.895 11.435 1.00 ± 0.02
head integration_builtins 10.994 ± 0.062 10.930 11.102 1.00
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 12.555 ± 0.239 12.313 13.110 1.01 ± 0.04
head keccak_integration_benchmark 12.467 ± 0.430 12.249 13.666 1.00
Command Mean [s] Min [s] Max [s] Relative
base linear_search 4.127 ± 0.013 4.111 4.150 1.00
head linear_search 4.145 ± 0.043 4.106 4.252 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 2.920 ± 0.030 2.903 3.004 1.00 ± 0.01
head math_cmp_and_pow_integration_benchmark 2.916 ± 0.006 2.903 2.925 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 2.676 ± 0.032 2.658 2.764 1.00
head math_integration_benchmark 2.685 ± 0.046 2.659 2.811 1.00 ± 0.02
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 2.351 ± 0.008 2.340 2.368 1.00
head memory_integration_benchmark 2.356 ± 0.011 2.341 2.380 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 2.670 ± 0.004 2.662 2.675 1.00
head operations_with_data_structures_benchmarks 2.675 ± 0.020 2.662 2.730 1.00 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 897.8 ± 8.6 893.2 921.6 1.00 ± 0.01
head pedersen 895.2 ± 4.2 892.2 906.0 1.00
Command Mean [s] Min [s] Max [s] Relative
base poseidon_integration_benchmark 1.617 ± 0.004 1.609 1.623 1.00
head poseidon_integration_benchmark 1.618 ± 0.014 1.609 1.656 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 3.117 ± 0.022 3.103 3.173 1.00
head secp_integration_benchmark 3.123 ± 0.011 3.110 3.143 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base set_integration_benchmark 1.569 ± 0.012 1.559 1.599 1.00
head set_integration_benchmark 1.577 ± 0.017 1.564 1.622 1.01 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 7.622 ± 0.019 7.608 7.672 1.00 ± 0.00
head uint256_integration_benchmark 7.602 ± 0.017 7.580 7.637 1.00

@pefontana pefontana enabled auto-merge June 12, 2023 18:44
@pefontana pefontana added this pull request to the merge queue Jun 12, 2023
Merged via the queue into main with commit 8e022a1 Jun 12, 2023
@pefontana pefontana deleted the add-proofmode-flags branch June 12, 2023 19:34
kariy pushed a commit to dojoengine/cairo-rs that referenced this pull request Jun 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pipelines This PR/issue is exclusively about improving our CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants