Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schnorr signature batch verification #2961

Merged
merged 5 commits into from
Sep 3, 2019
Merged

schnorr signature batch verification #2961

merged 5 commits into from
Sep 3, 2019

Conversation

garyyu
Copy link
Contributor

@garyyu garyyu commented Jul 23, 2019

To complete the schnorr signature batch verification feature.

The related PRs:

This should bring us over 50% performance improvement on schnorr signature verification:

if we batch with 1000 signatures, the average time is 40.5/78.5 = 51.6% compared to no batching.

@garyyu garyyu added this to the 2.x.x milestone Jul 23, 2019
Copy link
Member

@antiochp antiochp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - couple of minor comments.

chain/src/txhashset/txhashset.rs Outdated Show resolved Hide resolved
@@ -1323,28 +1323,39 @@ impl<'a> Extension<'a> {

let mut kern_count = 0;
let total_kernels = pmmr::n_leaves(self.kernel_pmmr.unpruned_size());
let mut tx_kernels: Vec<TxKernel> = Vec::with_capacity(1_024);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can simplify this by just using an iterator and chunks.
https://doc.rust-lang.org/std/primitive.slice.html#method.chunks

That way we don't need to hand code the chunking logic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for late response, let me finish this chunks today.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, the chunks is useful for splitting an existing big vector, but here I need create and fill a new vector, and I don't want to create a huge vector and later to split it with chunks. So I think it's better to keep it like this.

Pls let me know if thought wrongly.

let secp = secp.lock();

for tx_kernel in tx_kernels {
if tx_kernel.is_coinbase() && tx_kernel.fee != 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we split the fee and lock_height verification out from this batch signature verification?
There are some pending changes in #2859 that rework this and these don't need to be batched up.
We can just do 2 passes over the kernels -

  • verify fee and lock_height
  • then batch verify the signatures

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, I can split it out and will rework it.
BTW, when do you plan to complete #2859?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will try the splitting today and request a new review on this.

@quentinlesceller quentinlesceller changed the base branch from milestone/2.x.x to master July 24, 2019 15:41
@quentinlesceller
Copy link
Member

Rebased to master.

@garyyu
Copy link
Contributor Author

garyyu commented Aug 27, 2019

@antiochp Updated. Could you take a review again? thanks.

@garyyu
Copy link
Contributor Author

garyyu commented Aug 27, 2019

Oops, just saw the fee and lock_height checking has been removed from kernel verify function in #2859:

So, let me rework it again to remove the fee_height_verify function which I just split from origin verify function.

@garyyu
Copy link
Contributor Author

garyyu commented Aug 27, 2019

@antiochp I have a related question on https://github.com/mimblewimble/grin/pull/2859/files#r318043781 . Could you please have a look?

@antiochp
Copy link
Member

Testing this locally.

for x in &kernels {
x.verify()?;
}
TxKernel::batch_sig_verify(&kernels)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing we are fine just batching them all up like this in the context of a block.
But is there a limit to how many we can batch?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The limit should be big enough. At least 100,000 batching is OK:

running 1 test
Verifying aggsig batch of 100000
100000 signatures verified in 6408.601ms
test aggsig::tests::test_aggsig_batch ... ok

@antiochp
Copy link
Member

This looks significantly faster than 50% speedup to me. More like 5x -

20190829 18:13:56.905 DEBUG grin_chain::txhashset::txhashset - txhashset: verify_kernel_signatures: verified 5000 signatures
20190829 18:13:57.253 DEBUG grin_chain::txhashset::txhashset - txhashset: verify_kernel_signatures: verified 10000 signatures
20190829 18:13:57.599 DEBUG grin_chain::txhashset::txhashset - txhashset: verify_kernel_signatures: verified 15000 signatures
20190829 18:13:57.952 DEBUG grin_chain::txhashset::txhashset - txhashset: verify_kernel_signatures: verified 20000 signatures
20190829 18:13:58.301 DEBUG grin_chain::txhashset::txhashset - txhashset: verify_kernel_signatures: verified 25000 signatures
20190829 18:13:58.655 DEBUG grin_chain::txhashset::txhashset - txhashset: verify_kernel_signatures: verified 30000 signatures
20190829 18:13:59.003 DEBUG grin_chain::txhashset::txhashset - txhashset: verify_kernel_signatures: verified 35000 signatures
20190829 18:13:59.351 DEBUG grin_chain::txhashset::txhashset - txhashset: verify_kernel_signatures: verified 40000 signatures

Copy link
Member

@antiochp antiochp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question on batching without any limit for blocks.

Otherwise 👍 this runs great locally.

@DavidBurkett
Copy link
Contributor

Shouldn't be more than 50%, unless we were doing something really inefficient before. Fingers crossed that we were 😂

@antiochp antiochp merged commit 928279a into mimblewimble:master Sep 3, 2019
garyyu added a commit to gottstech/grin that referenced this pull request Sep 7, 2019
* schnorr signature batch verification

* Split the fee and lock_height verification out from the batch signature verification

* Remove the new added fee_height_verify to comply with mimblewimble#2859

* fix: the last n could not be leaf?
@garyyu garyyu deleted the schnorr-batch branch December 11, 2019 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants