Skip to content

Conversation

@Juan-M-V
Copy link
Contributor

@Juan-M-V Juan-M-V commented Jun 30, 2023

Add arbitrary and create a fuzzer using arbitrary for programs

Description

The arbitrary crate provides a derive macro to allow structs to be generated easily from random bytes. This is useful for proptests or structure aware fuzzers.
A feature was added to the VM and the felt crate to give the arbitrary derive to the Program and CairoRunConfig structs. Also a fuzzer using this feature was created.

Checklist

  • Linked to Github Issue
  • Unit tests added
  • Integration tests added.
  • This change requires new documentation.
    • Documentation has been added/updated.
    • CHANGELOG has been updated.

Base automatically changed from cairo-fuzzer to main June 30, 2023 18:36
@github-actions
Copy link

github-actions bot commented Jun 30, 2023

Benchmark Results for unmodified programs 🚀

Command Mean [s] Min [s] Max [s] Relative
base big_factorial 4.291 ± 0.034 4.270 4.383 1.01 ± 0.01
head big_factorial 4.255 ± 0.014 4.242 4.278 1.00
Command Mean [s] Min [s] Max [s] Relative
base big_fibonacci 3.656 ± 0.011 3.642 3.680 1.00 ± 0.00
head big_fibonacci 3.640 ± 0.011 3.621 3.655 1.00
Command Mean [s] Min [s] Max [s] Relative
base blake2s_integration_benchmark 11.016 ± 0.058 10.970 11.160 1.00
head blake2s_integration_benchmark 11.050 ± 0.083 10.940 11.194 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base compare_arrays_200000 3.971 ± 0.008 3.961 3.985 1.00 ± 0.00
head compare_arrays_200000 3.958 ± 0.014 3.941 3.982 1.00
Command Mean [s] Min [s] Max [s] Relative
base dict_integration_benchmark 2.612 ± 0.005 2.607 2.625 1.00
head dict_integration_benchmark 2.621 ± 0.005 2.615 2.633 1.00 ± 0.00
Command Mean [s] Min [s] Max [s] Relative
base field_arithmetic_get_square_benchmark 1.929 ± 0.005 1.923 1.937 1.00 ± 0.00
head field_arithmetic_get_square_benchmark 1.926 ± 0.005 1.921 1.934 1.00
Command Mean [s] Min [s] Max [s] Relative
base integration_builtins 9.786 ± 0.080 9.725 9.945 1.00 ± 0.01
head integration_builtins 9.751 ± 0.041 9.719 9.853 1.00
Command Mean [s] Min [s] Max [s] Relative
base keccak_integration_benchmark 11.085 ± 0.047 11.049 11.201 1.00
head keccak_integration_benchmark 11.125 ± 0.130 11.055 11.486 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base linear_search 3.947 ± 0.010 3.934 3.964 1.00
head linear_search 3.966 ± 0.007 3.958 3.981 1.00 ± 0.00
Command Mean [s] Min [s] Max [s] Relative
base math_cmp_and_pow_integration_benchmark 2.888 ± 0.009 2.877 2.905 1.00 ± 0.01
head math_cmp_and_pow_integration_benchmark 2.875 ± 0.014 2.862 2.902 1.00
Command Mean [s] Min [s] Max [s] Relative
base math_integration_benchmark 2.583 ± 0.007 2.574 2.596 1.00 ± 0.00
head math_integration_benchmark 2.580 ± 0.009 2.571 2.601 1.00
Command Mean [s] Min [s] Max [s] Relative
base memory_integration_benchmark 2.268 ± 0.006 2.262 2.282 1.00
head memory_integration_benchmark 2.277 ± 0.010 2.265 2.297 1.00 ± 0.01
Command Mean [s] Min [s] Max [s] Relative
base operations_with_data_structures_benchmarks 2.520 ± 0.004 2.515 2.527 1.00
head operations_with_data_structures_benchmarks 2.535 ± 0.014 2.524 2.569 1.01 ± 0.01
Command Mean [ms] Min [ms] Max [ms] Relative
base pedersen 862.5 ± 12.7 855.9 898.3 1.01 ± 0.01
head pedersen 855.1 ± 1.8 853.8 860.0 1.00
Command Mean [s] Min [s] Max [s] Relative
base poseidon_integration_benchmark 1.566 ± 0.005 1.560 1.576 1.00
head poseidon_integration_benchmark 1.567 ± 0.004 1.562 1.572 1.00 ± 0.00
Command Mean [s] Min [s] Max [s] Relative
base secp_integration_benchmark 2.991 ± 0.007 2.977 3.003 1.00
head secp_integration_benchmark 2.992 ± 0.007 2.984 3.010 1.00 ± 0.00
Command Mean [s] Min [s] Max [s] Relative
base set_integration_benchmark 1.489 ± 0.003 1.485 1.494 1.00 ± 0.00
head set_integration_benchmark 1.485 ± 0.002 1.482 1.488 1.00
Command Mean [s] Min [s] Max [s] Relative
base uint256_integration_benchmark 7.122 ± 0.028 7.086 7.170 1.00
head uint256_integration_benchmark 7.140 ± 0.022 7.113 7.181 1.00 ± 0.01

@Juan-M-V Juan-M-V added tests Implementation of tests coverage labels Jul 3, 2023
@pefontana
Copy link
Collaborator

pefontana commented Jul 4, 2023

@Juan-M-V please can you solve the conflicts and check the failing CI workflows
Also, add a line explaining the changes in the CHANGELOG.md and add a little description in the PR description

@codecov
Copy link

codecov bot commented Jul 7, 2023

Codecov Report

Merging #1306 (7ede34b) into main (b7ac1e8) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1306   +/-   ##
=======================================
  Coverage   97.88%   97.88%           
=======================================
  Files          89       89           
  Lines       35469    35469           
=======================================
  Hits        34719    34719           
  Misses        750      750           
Impacted Files Coverage Δ
felt/src/lib_lambdaworks.rs 97.79% <ø> (ø)
vm/src/cairo_run.rs 99.50% <ø> (ø)
vm/src/hint_processor/hint_processor_definition.rs 79.10% <ø> (ø)
vm/src/serde/deserialize_program.rs 97.25% <ø> (ø)
vm/src/types/instruction.rs 95.74% <ø> (ø)
vm/src/types/program.rs 99.47% <ø> (ø)
vm/src/types/relocatable.rs 99.85% <ø> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@pefontana pefontana added this pull request to the merge queue Jul 11, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to a conflict with the base branch Jul 11, 2023
@pefontana pefontana removed tests Implementation of tests coverage labels Jul 11, 2023
@pefontana pefontana enabled auto-merge July 11, 2023 20:33
@pefontana pefontana added this pull request to the merge queue Jul 11, 2023
Merged via the queue into main with commit 2d5b2cc Jul 11, 2023
@pefontana pefontana deleted the add-abitrary-fuzzing branch July 11, 2023 22:03
kariy pushed a commit to dojoengine/cairo-rs that referenced this pull request Jul 25, 2023
* Create fuzzer

* Remove cli

* Update fuzzer

* Change fuzzer name

* Add arbitrary

* Add fuzzer using arbitrary

* Add fuzzer

* Use run until steps

* Remove end

* Update changelog

* Update CHANGELOG.md

* Update CHANGELOG.md

* Change macro cfg

* Update bincode

* Update Arbitrary deps

* Check for std with arbitrary

* Update Cargo.toml

* Revert "Update bincode"

This reverts commit 6b5a95a.

* Revert to update bincode

* Fix arbitrary imports

* Run linter

* Run linter

---------

Co-authored-by: Juanma <[email protected]>
Co-authored-by: Pedro Fontana <[email protected]>
kariy pushed a commit to dojoengine/cairo-rs that referenced this pull request Jul 25, 2023
* Create fuzzer

* Remove cli

* Update fuzzer

* Change fuzzer name

* Add arbitrary

* Add fuzzer using arbitrary

* Add fuzzer

* Use run until steps

* Remove end

* Update changelog

* Update CHANGELOG.md

* Update CHANGELOG.md

* Change macro cfg

* Update bincode

* Update Arbitrary deps

* Check for std with arbitrary

* Update Cargo.toml

* Revert "Update bincode"

This reverts commit 6b5a95a.

* Revert to update bincode

* Fix arbitrary imports

* Run linter

* Run linter

---------

Co-authored-by: Juanma <[email protected]>
Co-authored-by: Pedro Fontana <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants