-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Streamline macro syntax #169
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Details: - Simplified the implementation of `BatchedRelaxedR1CSSNARKTrait` by replacing usage of `zip_with_iter` and related macros with `zip_with_fn`. - Eliminated `zip_with_iter`, `zip_with_par_iter`, `zip_with_into_iter`, and `zip_with_into_par_iter` macros from the codebase, reducing repetition and enhancing code organization.
huitseeker
force-pushed
the
streamlining
branch
3 times, most recently
from
December 9, 2023 14:41
ba22019
to
3189aee
Compare
- Replaced the `zip_with_fn!` function with the `zip_with!` function in various files, - Refactored `zip_with` and `zip_with_for_each` macros in `src/spartan/macros.rs` for improved handling of iterator projection specification. - Consolidated `zip_all_with_fn` macro's functionality into `zip_all` and eliminated unused macros.
huitseeker
force-pushed
the
streamlining
branch
from
December 9, 2023 14:43
3189aee
to
106e7ad
Compare
Merged
porcuquine
previously approved these changes
Dec 9, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a natural next (perhaps final) evolution of these macros.
* refactor: removed needless copies of `R1CSShape` - Modified the `primary_r1cs_shapes` function in `supernova/mod.rs` to return a vector of references, improving memory efficiency. - Use those `primary_r1cs_shapes` in the `setup` and `prove` methods of `CompressedSNARK`. - Altered the `BatchedRelaxedR1CSSNARKTrait` trait functions in `traits/snark.rs` to accept vector of references instead of reference, * refactor: Refactor polynomial size assertion checks in sumcheck.rs - Refactors the length assertion checks in the `prove_quad_batch` and `prove_cubic_with_additive_term_batch` methods in `src/spartan/sumcheck.rs`. - Simplified the code by replacing multiple `assert_eq` calls with a loop.
huitseeker
force-pushed
the
streamlining
branch
from
December 9, 2023 18:41
f0aa0a0
to
ed88500
Compare
- Removed the "nested_tuple" macro from the "src/spartan/macros.rs" file.
huitseeker
force-pushed
the
streamlining
branch
from
December 9, 2023 18:57
bbd1e8c
to
8591f9f
Compare
porcuquine
approved these changes
Dec 9, 2023
github-merge-queue bot
pushed a commit
that referenced
this pull request
Dec 11, 2023
* skeleton of what is to be done * wip inner and outer sumchecks done * fix borrow checker issues * batched spartan prove done * Small improvements - Irregular shape yields error - Remove (W,E) clones - Use Math module for log2 - Generate tau using only 1 challenge - Remove some `clone()` - Comments * Fix evaluation points * Implement verifier * `CompressedSNARK` implementation * Tests fixed, failing * fix sumcheck, test passes * bound -> bind * remove todo * add test * Document BatchedRelaxedR1CSSNARKTrait and SuperNova CompressedSNARK. * working * - Remove debug code - Add Sumcheck::verify_batch - Uniformize batching of evals - Comments * Adress review comments - add itertools for multiunzip - avoid clones - avoid recomputing sizes for asserts * Fix asserts * batched ppsnark * Fix scaling * - Fix padding of poly_W - Optimized evaluation - use batch_diff_size - use multiunzip * rebase fixes * Only create taus once. * Avoid clone by not allocating PowPolynomial. * Riff on review. * refactor: Implement `.zip_eq()` for equal length iterator safety - Replaced `.zip()` with `.zip_eq()` across various files including `sumcheck.rs`, `mod.rs`, `batched_ppsnark.rs`, `batched.rs` and `snark.rs`, ensuring equal length enforcement in iterations and reducing out-of-bound errors. - Updated a multiunzip resulting in a double iteration in batched_ppsnark::prove. * add benchmark * fix witnes * fix: clean up iterators * feat: Introduce new utility macros in spartan module - Introduced a macro `zip_with` in `src/spartan/mod.rs` to simplify the process of zipping multiple iterators and applying a single parallel function. - Developed additional macros `nested_tuple` and `nested_idents` for detailed nested tuple patterns and identifiers respectively. - Implemented `zip_all` macro that acts as a fold-right zipping operator for all given expressions. * refactor: Refactor batched.rs to enhance code readability - Refactored `src/spartan/batched.rs` to use `zip_with()` macro improving the readability, simplification, and consistent usage in the code. - Inserted comments highlighting sections marked for review or possible future code simplification. * refactor: use zip_with! in batch_ppsnark - Updated `batched_ppsnark.rs` in the `src/spartan` directory with uses of zip_with macro * ignore local macro doctest * Use zip_with. * Implement and use zip_with_for_each. * Dubious double zip_with. * More zip_with. * Small fixes - `zip_with` for `batched_ppsnark` - Complete comment of `BatchedRelaxedR1CSSNARK` - Restore Abomonation bounds - Fix nits in `multilinear.rs` * refactor: more zip_with instances (#153) * refactor: more instances of zip_with * Supernova: Some rewrites (#154) * refactor: more zip_with * feat: Implement `FromIterator` for polynomial types - Enhanced `EqPolynomial`, `MultilinearPolynomial`, and `SparsePolynomial` with `FromIterator` implementation. * refactor: use instaces of Polynomial Fromiterator where possible * refactor: remove unused impls of FromIterator * refactor: address review comments * refactor: enhance scope visibility scope in Spartan and Supernova modules (#156) Absolute visibility modifiers tend to not inadvertently make things public when you move code, contrarily to (super). - Changed the visibility scope of several fields and methods within multiple substrates in the `ppsnark.rs` file to be within the `crate::spartan`. - Altered the module import path in the `circuit.rs` file from `super` to `crate::supernova`. - Modified the visibility of the `WitnessBoundSumcheck` in `batched_ppsnark.rs`, the visibility scope is now set to `crate::spartan`. - Scoped `batch_eval_prove` and `batch_eval_verify` functions within `crate::spartan` in `snark.rs`. * Avoid EqPolynomial when only for evals. * refactor: a few more instances of EqPolynomial::evals_from_points - in `multilinear.rs` and `ipa_pc.rs`, replaces the previous method `EqPolynomial::new(r.to_vec()).evals()` with a more efficient `EqPolynomial::evals_from_points(r)`. * doc: fix a few nits (#160) * refactor: rename powers -> squares - Renamed the `powers` method to `squares` in `PowPolynomial` struct within `src/spartan/polys/power.rs` and changed its visibility level. * fix: comment * fix: comment of batch * derive fixes * fix: compressed snark benchmark - Introduced `compressed-snark-supernova` within the `Cargo.toml` file. - fixed Rust * ci: Add trigger for PRs to batched_spartan (#165) * More zip with (#158) * Expand zip_with macros to accommodate more use. * Clippy. * Move Spartan macros to own module. * Only use zip_with_fn to implement convenience macros. * Use convenience zip_with macros more. * Remove for_each variants of zip_with macros. * Remove flat_map variants of zip_with. --------- Co-authored-by: porcuquine <[email protected]> * Introduce proper `MaskedEq` poly (#162) * Introduce proper `MaskedEq` poly * refactor: Refactor MaskedEq poly for improved testing and performance - Introduce lifetime parameters to `MaskedEqPolynomial` struct to improve memory management. - Modify the `MaskedEqPolynomial` creation process so it now requires a reference to an existing `EqPolynomial`. - Adapt corresponding tests to reflect the above changes. * Fix fmt & derive debug --------- Co-authored-by: François Garillot <[email protected]> * Restore zip_with_for_each. (#166) Co-authored-by: porcuquine <[email protected]> * fix benchmark (#168) * Sumcheck update (#163) * Include number of instances in transcript * Clarify Sumcheck and verifier * Fix clippy * Streamline macro syntax (#169) * refactor: Refactor Spartan code for streamlined use of `zip_with_fn` Details: - Simplified the implementation of `BatchedRelaxedR1CSSNARKTrait` by replacing usage of `zip_with_iter` and related macros with `zip_with_fn`. - Eliminated `zip_with_iter`, `zip_with_par_iter`, `zip_with_into_iter`, and `zip_with_into_par_iter` macros from the codebase, reducing repetition and enhancing code organization. * refactor: Refactor zip functions - Replaced the `zip_with_fn!` function with the `zip_with!` function in various files, - Refactored `zip_with` and `zip_with_for_each` macros in `src/spartan/macros.rs` for improved handling of iterator projection specification. - Consolidated `zip_all_with_fn` macro's functionality into `zip_all` and eliminated unused macros. * fix: remove brackets * Small fixes (#170) * refactor: removed needless copies of `R1CSShape` - Modified the `primary_r1cs_shapes` function in `supernova/mod.rs` to return a vector of references, improving memory efficiency. - Use those `primary_r1cs_shapes` in the `setup` and `prove` methods of `CompressedSNARK`. - Altered the `BatchedRelaxedR1CSSNARKTrait` trait functions in `traits/snark.rs` to accept vector of references instead of reference, * refactor: Refactor polynomial size assertion checks in sumcheck.rs - Refactors the length assertion checks in the `prove_quad_batch` and `prove_cubic_with_additive_term_batch` methods in `src/spartan/sumcheck.rs`. - Simplified the code by replacing multiple `assert_eq` calls with a loop. * refactor: Remove "nested_tuple" macro from spartan macros - Removed the "nested_tuple" macro from the "src/spartan/macros.rs" file. * refactor: pedantic calls in tests --------- Co-authored-by: Matej Penciak <[email protected]> Co-authored-by: Adrian Hamelink <[email protected]> Co-authored-by: porcuquine <[email protected]> Co-authored-by: Hanting Zhang <[email protected]> Co-authored-by: Samuel Burnham <[email protected]> Co-authored-by: porcuquine <[email protected]> Co-authored-by: Adrian Hamelink <[email protected]> Co-authored-by: Matej Penciak <[email protected]>
huitseeker
added a commit
to huitseeker/Nova
that referenced
this pull request
Jan 10, 2024
- Created a new file `macros.rs` with the implementation of `zip_with` and `zip_with_for_each` macros providing syntactic sugar for zipWith patterns. - zipWith patterns implemented through the use of the zip_with! macros now resolve to the use of zip_eq, a variant of zip that panics when the iterator arguments are of different length, - the zip_eq implementation is the native rayon one for parallel iterators, and the one from itertools (see below) for the sequential ones, - Optimized and refactored functions like `batch_eval_prove` and `batch_eval_verify` in `snark.rs`, methods inside `PolyEvalWitness` and `PolyEvalInstance` in `mod.rs`, and multiple functions in `multilinear.rs` through the use of implemented macros. - Introduced the use of itertools::Itertools in various files to import the use of zip_eq on sequential iterators. - Made use of the Itertools library for refactoring and optimizing computation in `sumcheck.rs` and `eq.rs` files. This backports (among others) content from the following Arecibo PRS: - lurk-lang/arecibo#149 - lurk-lang/arecibo#158 - lurk-lang/arecibo#169 Co-authored-by: porcuquine <[email protected]>
srinathsetty
pushed a commit
to microsoft/Nova
that referenced
this pull request
Jan 10, 2024
- Created a new file `macros.rs` with the implementation of `zip_with` and `zip_with_for_each` macros providing syntactic sugar for zipWith patterns. - zipWith patterns implemented through the use of the zip_with! macros now resolve to the use of zip_eq, a variant of zip that panics when the iterator arguments are of different length, - the zip_eq implementation is the native rayon one for parallel iterators, and the one from itertools (see below) for the sequential ones, - Optimized and refactored functions like `batch_eval_prove` and `batch_eval_verify` in `snark.rs`, methods inside `PolyEvalWitness` and `PolyEvalInstance` in `mod.rs`, and multiple functions in `multilinear.rs` through the use of implemented macros. - Introduced the use of itertools::Itertools in various files to import the use of zip_eq on sequential iterators. - Made use of the Itertools library for refactoring and optimizing computation in `sumcheck.rs` and `eq.rs` files. This backports (among others) content from the following Arecibo PRS: - lurk-lang/arecibo#149 - lurk-lang/arecibo#158 - lurk-lang/arecibo#169 Co-authored-by: porcuquine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The Problem
The current macro system requires learning what 7 auxiliary macros do:
zip_with, zip_with_for_each, zip_with_into_par_iter, zip_with_iter, zip_with_par_iter, zip_with_par_iter_mut_for_each, zip_with_into_iter
.This PR
Switches us to a mode in which the user has only 2 macros to learn about :
zip_with
andzip_with_for_each
.This further documents them as such:
and:
Everything is refactored to use one of these two macros. While there are more complex forms to these macros, the user doesn't need to import them or know about them. The implementations are also simplified.
Details
Also contains #170 because stacked PRs in Git are a mess.