Separate block confirmations #501

haerdib · 2021-11-10T12:31:31Z

Seems to work quite well: For every parentchain block we get 20 sidechain block events.

Introduces the following changes:

exchanges call- and block confirmation with parentchain and sidechain confirmations
removes IpfsHash Update Event (was removed completely from pallet teerex)
updates substrate version to the same of the integritee-node (https://github.com/paritytech/substrate.git?branch=master#b391b82954ad95a927a921035e3017c4a0aad516), wasn't compiling otherwise
cleans up shielding call execution call (at least a little)
temporarily introduces a new file for Merkel root calculation. This can be removed once we update to a substrate commit >= 23rd September

closes #457

haerdib · 2021-11-10T13:07:46Z

core-primitives/stf-executor/src/executor.rs

 		account: AccountId,
 		amount: Amount,
 		shard: &ShardIdentifier,
-		calls: &mut Vec<OpaqueCall>,


Removed this input parameter, as there will never(!) be a new OpaqueCall introduced during Stf::execute of a shielding extrsinic. (New OpaqueCall is only issued during execute when calling "unshield")

Thanks for clarifying this ☺️

Haha, good catch. 🛩️

enclave-runtime/src/lib.rs

haerdib · 2021-11-10T13:10:43Z

enclave-runtime/Cargo.toml

 sp-utils = { version = "4.0.0-dev", default-features = false, git = "https://github.com/paritytech/substrate.git", branch = "master" }
 sp-version = { version = "4.0.0-dev", default-features = false, git = "https://github.com/paritytech/substrate.git", branch = "master" }
 sp-application-crypto = { version = "4.0.0-dev", default-features = false, git = "https://github.com/paritytech/substrate.git", branch = "master" }
+#beefy-merkle-tree = { version = "4.0.0-dev", default-features = false, git = "https://github.com/paritytech/substrate.git", branch = "master", features = "keccak" }


That's the ugly part here: substrate actually has a no-std compatible merkle tree primitives crate: https://github.com/paritytech/substrate/tree/master/frame/beefy-mmr/primitives

However, this was introduced on 23rd of September. We are not yet on this substrate commit. Can you agree with this temporary solution and to remove it once we are on this commit? If yes, I'll create an appropriate issue ofc.

If it's well documented and even has a corresponding issue, that's totally fine for me 👍

I think it's fine too. 👍

haerdib · 2021-11-10T13:11:40Z

enclave-runtime/src/beefy_merkle_tree.rs

@@ -0,0 +1,229 @@
+// This file is part of Substrate.


This file was copy pasted from https://github.com/paritytech/substrate/blob/master/frame/beefy-mmr/primitives/src/lib.rs. Can be removed once we are above commit paritytech/substrate@65ac6f3

murerfel · 2021-11-10T13:35:27Z

enclave-runtime/src/lib.rs

-	let xt_block = [TEEREX_MODULE, BLOCK_CONFIRMED];
-	let opaque_call =
-		OpaqueCall::from_tuple(&(xt_block, shard, block_hash, state_hash_new.encode()));
+	let opaque_call = proposed_sidechain_block_extrinsic(shard, block_hash);


so we now use the sidechain block hash instead of the state hash for this extrinsic?

Not exactly. State hash has been removed from the extrinsic as it didn't have any use case at all (#457)

murerfel · 2021-11-10T13:36:55Z

enclave-runtime/src/lib.rs

+/// Creates a proposed_sidechain_block extrinsic for a given shard id and sidechain block hash.
+fn proposed_sidechain_block_extrinsic(shard_id: ShardIdentifier, block_hash: H256) -> OpaqueCall {
+	OpaqueCall::from_tuple(&([TEEREX_MODULE, PROPOSED_SIDECHAIN_BLOCK], shard_id, block_hash))
+}


Also here, I'd prefer the create_ in front. In a future refactoring, we could put these into the extrinsics factory component or something similar?

Already changed. :) Yes, definitely - but I think the whole lib.rs (just like in PR #500 but also for parentchain syncing) should be refactored (but that's your call).. I didn't want to start something that will be changed in the future anyway. If you have a place they belong to already, I'll gladly move them.

murerfel · 2021-11-10T13:40:51Z

enclave-runtime/src/tests.rs

+fn empty_extrinsic_vec_gives_zero_merkle_root() {
+	// given
+	let block_hash = H256::from([1; 32]);
+	let extrinsics = Vec::new();
+	let expected_call =
+		([TEEREX_MODULE, PROCESSED_PARENTCHAIN_BLOCK], block_hash, H256::default()).encode();
+
+	// when
+	let call = crate::processed_parentchain_block_extrinsic(block_hash, extrinsics);
+
+	// then
+	assert_eq!(call.0, expected_call);
+}
+
+fn some_extrinsics_vec_give_non_zero_merkle_root() {
+	// given
+	let block_hash = H256::from([1; 32]);
+	let extrinsics = vec![H256::from([4; 32]), H256::from([9; 32])];
+	let zero_root_call =
+		([TEEREX_MODULE, PROCESSED_PARENTCHAIN_BLOCK], block_hash, H256::default()).encode();
+
+	// when
+	let call = crate::processed_parentchain_block_extrinsic(block_hash, extrinsics);
+
+	// then
+	assert_ne!(call.0, zero_root_call);
+}


Always nice to have tests for the changes you make 🥳

murerfel · 2021-11-10T13:43:11Z

enclave-runtime/src/lib.rs

+fn hash_of<T: Encode>(xt: T) -> H256 {
+	blake2_256(&xt.encode()).into()


This is a function that we have duplicated multiple times I believe. We should at some point put it in a primitives crate where it can be properly shared.

Aye.. But I didn't want to touch too much code, in fear of the rebase conflicts 😨

I think we should simply implement hash on the extrinsic in the api-client. 😄

murerfel

LGTM - and if the checks pass, it should be fine ☺️ thanks for your perseverance when it comes to good comments 🚀

clangenb

Looks good to me. 👍

clangenb · 2021-11-11T01:18:32Z

core-primitives/stf-executor/src/executor.rs

 		account: AccountId,
 		amount: Amount,
 		shard: &ShardIdentifier,
-		calls: &mut Vec<OpaqueCall>,


Haha, good catch. 🛩️

clangenb · 2021-11-11T01:19:37Z

enclave-runtime/Cargo.toml

 sp-utils = { version = "4.0.0-dev", default-features = false, git = "https://github.com/paritytech/substrate.git", branch = "master" }
 sp-version = { version = "4.0.0-dev", default-features = false, git = "https://github.com/paritytech/substrate.git", branch = "master" }
 sp-application-crypto = { version = "4.0.0-dev", default-features = false, git = "https://github.com/paritytech/substrate.git", branch = "master" }
+#beefy-merkle-tree = { version = "4.0.0-dev", default-features = false, git = "https://github.com/paritytech/substrate.git", branch = "master", features = "keccak" }


I think it's fine too. 👍

clangenb · 2021-11-11T01:24:14Z

enclave-runtime/src/lib.rs

 		if let Err(e) = validator.submit_simple_header(
 			validator.num_relays(),
-			signed_block.block.header().clone(),
+			block.header().clone(),


nice optimization.

clangenb · 2021-11-11T01:28:04Z

enclave-runtime/src/lib.rs

+fn hash_of<T: Encode>(xt: T) -> H256 {
+	blake2_256(&xt.encode()).into()


I think we should simply implement hash on the extrinsic in the api-client. 😄

haerdib force-pushed the separate-block-confirmations branch from 9e45e6a to 386000b Compare November 10, 2021 12:49

haerdib self-assigned this Nov 10, 2021

haerdib commented Nov 10, 2021

View reviewed changes

haerdib requested review from clangenb and murerfel and removed request for clangenb November 10, 2021 13:24

murerfel reviewed Nov 10, 2021

View reviewed changes

enclave-runtime/src/lib.rs Outdated Show resolved Hide resolved

haerdib added 17 commits November 10, 2021 14:52

renmae call_confirmmed & block_confirmed

20d4541

rename CallConfirmed Event to ProcessedParentchainBlock

e966199

fix comment

70538fe

update to new node

63773d9

[shield_funds] remove unused passed Vec<OpaqueCalls>

8e63037

remove outdated ipfshash event

9a6dc9f

add ProposedSidechainBlock event and fix compile errors

0684abc

restructure shielding fund exeuction (remove unnecessary input values)

7be4d62

add merkle root creation to processed parentchain block

19aaeb1

add unit tests to merkle function

0ec04d8

clean up confirmation makings

2496ee1

change integritee runtime branch back to master

47fe70f

remove comment

b0614da

add . to comment ;)

22a3d39

[GA] udpate to latest node artifact

4d9fd34

add create_ to block confirmation creation functions

e38f887

add some better test names

7ac89fe

haerdib commented Nov 10, 2021

View reviewed changes

update cargo.lock

5e49a19

haerdib force-pushed the separate-block-confirmations branch from f1773ad to 5e49a19 Compare November 10, 2021 14:04

rename _block_extrinsic to _block_call (extrinsics are created later on)

6a18f1a

murerfel reviewed Nov 10, 2021

View reviewed changes

murerfel approved these changes Nov 10, 2021

View reviewed changes

clangenb approved these changes Nov 11, 2021

View reviewed changes

haerdib merged commit e08bf0c into master Nov 11, 2021

haerdib deleted the separate-block-confirmations branch November 11, 2021 07:33

haerdib mentioned this pull request Nov 12, 2021

Fix event decoding in cli + remove timeout in demos #507

Merged

		fn hash_of<T: Encode>(xt: T) -> H256 {
		blake2_256(&xt.encode()).into()

Separate block confirmations #501

Separate block confirmations #501

Uh oh!

Conversation

haerdib commented Nov 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

haerdib Nov 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

haerdib Nov 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

murerfel left a comment

Choose a reason for hiding this comment

Uh oh!

clangenb left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

haerdib commented Nov 10, 2021 •

edited

Loading

haerdib Nov 10, 2021 •

edited

Loading

haerdib Nov 10, 2021 •

edited

Loading