-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Add comparative GPU benchmarks on merge_group
#238
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
samuelburnham
pushed a commit
to samuelburnham/arecibo
that referenced
this pull request
Jan 5, 2024
* Digest simplifications (lurk-lang#238) * remove unused digest computations * avoid a verifier having to recompute a digest * update crate version Restore digest computation and fix API inconsistency (lurk-lang#242) * Revert "Digest simplifications (lurk-lang#238)" This reverts commit 71ecb66. * upgrade neptune * make the public interface uniform wrt refs vs. copies * simplify prove_step * refactor: Adapt supernova RecursiveSNARK to Nova API changes - Updated `RecursiveSNARK` struct in `supernova/mod.rs` to include `z0_primary` and `z0_secondary` fields, simplifying method parameters. - Refactored `prove_step` method in `RecursiveSNARK` struct to leverage the new instance variables, `z0_primary` and `z0_secondary`, - Replaced all usages of `z0_primary` and `z0_secondary` in function calls with their respective instance variables. --------- Co-authored-by: Srinath Setty <[email protected]>
samuelburnham
force-pushed
the
gpu-bench
branch
4 times, most recently
from
January 8, 2024 22:21
961fe26
to
b381177
Compare
samuelburnham
force-pushed
the
gpu-bench
branch
from
January 8, 2024 22:36
b381177
to
a621594
Compare
This was referenced Jan 8, 2024
samuelburnham
force-pushed
the
gpu-bench
branch
from
January 8, 2024 23:53
a621594
to
8af5dc3
Compare
samuelburnham
force-pushed
the
gpu-bench
branch
from
January 8, 2024 23:57
8af5dc3
to
ef6253d
Compare
samuelburnham
changed the title
ci: Add comparative benchmarks on
ci: Add comparative GPU benchmarks on Jan 8, 2024
merge_group
(WIP)merge_group
huitseeker
approved these changes
Jan 9, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, this looks like a lot of progress, let's merge and iron out the kinks.
This was referenced Jan 10, 2024
huitseeker
pushed a commit
to huitseeker/arecibo
that referenced
this pull request
Jan 14, 2024
* remove unused digest computations * avoid a verifier having to recompute a digest * update crate version
huitseeker
pushed a commit
to huitseeker/arecibo
that referenced
this pull request
Jan 14, 2024
* Revert "Digest simplifications (lurk-lang#238)" This reverts commit 71ecb66. * upgrade neptune * make the public interface uniform wrt refs vs. copies * simplify prove_step
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
GPU benchmarks for PR regression testing
recursive-snark
benchmark group and ID to enable formatting withcriterion-table
. This will be extended tocompressed-snark
and their supernova counterparts when ready.criterion-table
output in a621594#commitcomment-136557829, and the basic logic works. Unfortunately, we're limited by the fact that the base benchmarks ondev
don't have the same formatting or parameters as this PR, so it's impossible to demo exactly what it will look like beyond the proposal below.bc
on GPU runners argumentcomputer/gh-actions-runner#25)Proposed
criterion-table
output:Benchmarks
Table of Contents
Benchmark Results
RecursiveSNARK
ref=ae521be709871f94a622c85b0e5d6e8f776be144
ref=ace6724dc77667506dbcd1122b9851bd6edb25d2
Prove-StepCircuitSize-0
29.28 ms
(✅ 1.00x)28.71 ms
(✅ 1.02x faster)Verify-StepCircuitSize-0
19.48 ms
(✅ 1.00x)19.10 ms
(✅ 1.02x faster)Prove-StepCircuitSize-6559
35.57 ms
(✅ 1.00x)35.27 ms
(✅ 1.01x faster)Verify-StepCircuitSize-6559
23.84 ms
(✅ 1.00x)23.89 ms
(✅ 1.00x slower)Made with criterion-table
Design notes
For the other benchmarks which have more than one group, they would each appear in their own table (e.g.
CompressedSNARK
andCompressedSNARK-Commitments
). This layout will be shown as well when implemented.Some other display options:
StepCircuitSize
into its own group, which is the existing behavior. This would proliferate the number of benchmark tables to a large degree if we are displaying four different sets of benchmarks in one.md
fileProve
andVerify
benchmarks into their own table or group them together. This would require some finagling since the benchmarks are currently ordered sequentially in the order they are run.Future work
Helps #243