refactor: l1 l2 messages cleanup by benesjan · Pull Request #5270 · AztecProtocol/aztec-packages

benesjan · 2024-03-17T15:42:41Z

Fixes #5264

benesjan · 2024-03-17T15:42:54Z

feat: returning non-nullified messages only #5278
refactor: nuking L1 block number from L2 block #5275
refactor: l1 l2 messages cleanup #5270 👈
refactor: removing redundant receipts check #5271 : 1 other dependent PR (#5272 )
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @benesjan and the rest of your teammates on Graphite

l1-contracts/src/core/interfaces/messagebridge/IInbox.sol

l1-contracts/test/portals/TokenPortal.t.sol

LHerskind

I think the issues are related to the empty tree values. Seems like it is the tests that need the message boxes, and some of the cleanup here have broken it 🤔

yarn-project/circuit-types/src/interfaces/block_number.ts

yarn-project/circuit-types/src/messaging/inbox_leaf.ts

yarn-project/circuit-types/src/messaging/l1_actor.ts

This reverts commit 4d3722f.

benesjan · 2024-03-19T10:40:23Z

yarn-project/archiver/src/archiver/data_retrieval.ts

+   * Blocknumber of the last L1 block from which we obtained data.
   */
-  nextEthBlockNumber: bigint;
+  lastProcessedL1BlockNumber: bigint;


Did this change because in places where we consume the value we need the last processed block num and not the next one after it so I think it makes sense to pass the value around in that format.

benesjan · 2024-03-19T10:41:40Z

yarn-project/circuit-types/src/interfaces/aztec-node.ts

   */
-  getL1ToL2MessageIndexAndSiblingPath(
-    blockNumber: BlockNumber,
+  getL1ToL2MessageMembershipWitness(


Did this change because the oracle which calls this func is called getL1ToL2MessageMembershipWitness as well so it makes sense to have it consistent.

benesjan · 2024-03-19T10:42:38Z

yarn-project/circuit-types/src/interfaces/aztec-node.ts

+   * @param l1ToL2Message - The L1 to L2 message to check.
+   * @returns Whether the message is synced and ready to be included in a block.
+   */
+  isL1ToL2MessageSynced(l1ToL2Message: Fr): Promise<boolean>;


Added this func so that I can keep on polling for the message to be available instead of having a hardcoded sleep timeout and hoping that the archiver manages to catch up in that time. Should result in more robust tests.

benesjan · 2024-03-19T10:43:42Z

yarn-project/archiver/src/archiver/memory_archiver_store/l1_to_l2_message_store.ts

      }
    }
-    throw new Error(`L1 to L2 message index not found in the store for message ${l1ToL2Message.toString()}`);
+    return undefined;


Returning undefined now because that is how other endpoints returning indices behave and it makes it more convenient for the implementation of the new isL1ToL2MessageSynced func on aztec node.

AztecBot · 2024-03-19T11:01:19Z

Benchmark results

Metrics with a significant change:

circuit_simulation_time_in_ms (public-kernel-app-logic): 530 (+19%)
circuit_simulation_time_in_ms (public-kernel-tail): 203 (+17%)
tx_sequencer_processing_time_ms (1): 1,452 (+17%)
l2_block_building_time_in_ms (32): 85,632 (+25%)
l2_block_building_time_in_ms (64): 174,988 (+29%)
l2_block_public_tx_process_time_in_ms (32): 56,374 (+42%)
l2_block_public_tx_process_time_in_ms (64): 118,008 (+49%)

Detailed results

All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.

This benchmark source data is available in JSON format on S3 here.

Values are compared against data from master at commit edb8c679 and shown if the difference exceeds 1%.

L2 block published to L1

Each column represents the number of txs on an L2 block published to L1.

Metric	8 txs	32 txs	64 txs
l1_rollup_calldata_size_in_bytes	5,668	18,820	36,356
l1_rollup_calldata_gas	66,388	239,032	469,832
l1_rollup_execution_gas	647,486	929,390	1,306,012
l2_block_processing_time_in_ms	1,347 (+4%)	5,110 (+11%)	9,784 (+4%)
note_successful_decrypting_time_in_ms	166 (-11%)	543 (+2%)	1,020 (+4%)
note_trial_decrypting_time_in_ms	71.1 (-21%)	64.3 (+16%)	56.1 (-53%)
l2_block_building_time_in_ms	18,136	⚠️ 85,632 (+25%)	⚠️ 174,988 (+29%)
l2_block_rollup_simulation_time_in_ms	8,090 (-1%)	29,203 (+2%)	56,881 (+1%)
l2_block_public_tx_process_time_in_ms	10,025	⚠️ 56,374 (+42%)	⚠️ 118,008 (+49%)

L2 chain processing

Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric	5 blocks	10 blocks
node_history_sync_time_in_ms	13,768 (-1%)	27,381
note_history_successful_decrypting_time_in_ms	1,198 (-4%)	2,382 (-1%)
note_history_trial_decrypting_time_in_ms	64.4 (-33%)	113 (-3%)
node_database_size_in_bytes	19,083,344	35,721,296
pxe_database_size_in_bytes	29,859	59,414

Circuits stats

Stats on running time and I/O sizes collected for every circuit run across all benchmarks.

Circuit	circuit_simulation_time_in_ms	circuit_input_size_in_bytes	circuit_output_size_in_bytes
private-kernel-init	279 (-1%)	44,366	28,244
private-kernel-ordering	212 (-1%)	52,868	14,326
base-parity	1,804	128	311
base-rollup	721	165,787	925
root-parity	1,629 (+3%)	1,244	311
root-rollup	51.6 (-18%)	4,487	725 (-8%)
private-kernel-inner	636 (-1%)	73,771	28,244
public-kernel-app-logic	⚠️ 530 (+19%)	35,260	28,215
public-kernel-tail	⚠️ 203 (+17%)	40,926	28,215
merge-rollup	8.40	2,696	925

Tree insertion stats

The duration to insert a fixed batch of leaves into each tree type.

Metric	1 leaves	16 leaves	64 leaves	128 leaves	512 leaves	1024 leaves	2048 leaves	4096 leaves	32 leaves
batch_insert_into_append_only_tree_16_depth_ms	10.1 (+1%)	16.0	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_count	16.8	31.6	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_16_depth_hash_ms	0.589 (+1%)	0.496	N/A	N/A	N/A	N/A	N/A	N/A	N/A
batch_insert_into_append_only_tree_32_depth_ms	N/A	N/A	47.5 (+3%)	73.6 (+2%)	239 (+3%)	443	911 (+3%)	1,739 (+1%)	N/A
batch_insert_into_append_only_tree_32_depth_hash_count	N/A	N/A	96.0	159	543	1,055	2,079	4,127	N/A
batch_insert_into_append_only_tree_32_depth_hash_ms	N/A	N/A	0.488 (+3%)	0.455 (+2%)	0.436 (+3%)	0.415	0.432 (+3%)	0.417 (+1%)	N/A
batch_insert_into_indexed_tree_20_depth_ms	N/A	N/A	54.2 (-1%)	111 (+4%)	348 (+3%)	659	1,442 (+10%)	2,835 (+9%)	N/A
batch_insert_into_indexed_tree_20_depth_hash_count	N/A	N/A	104	207	691	1,363	2,707	5,395	N/A
batch_insert_into_indexed_tree_20_depth_hash_ms	N/A	N/A	0.483 (-1%)	0.501 (+4%)	0.475 (+4%)	0.454	0.501 (+10%)	0.494 (+9%)	N/A
batch_insert_into_indexed_tree_40_depth_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	62.5 (+2%)
batch_insert_into_indexed_tree_40_depth_hash_count	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	109
batch_insert_into_indexed_tree_40_depth_hash_ms	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	0.547 (+2%)

Miscellaneous

Transaction sizes based on how many contract classes are registered in the tx.

Metric	0 registered classes
tx_size_in_bytes	22,012

Transaction processing duration by data writes.

Metric	0 new note hashes	1 new note hashes
tx_pxe_processing_time_ms	3,189 (-2%)	1,727 (-1%)

Metric	0 public data writes	1 public data writes
tx_sequencer_processing_time_ms	12.0 (+9%)	⚠️ 1,452 (+17%)

github-actions · 2024-03-19T11:04:59Z

Changes to circuit sizes

Generated at commit: 6ee3a8d9c6e59d65e2f83efd4052fb16e7b862ea, compared to commit: acffa7b4d41496dea0d16fd94ab98a1a977d14a8

🧾 Summary (100% most significant diffs)

Program	ACIR opcodes (+/-)	%	Circuit size (+/-)	%
rollup_root	-1,098 ✅	-50.14%	-26,213 ✅	-25.18%

Full diff report 👇

Program	ACIR opcodes (+/-)	%	Circuit size (+/-)	%
rollup_root	1,092 (-1,098)	-50.14%	77,898 (-26,213)	-25.18%

LHerskind

Good stuff. 🚢

benesjan mentioned this pull request Mar 17, 2024

refactor: old inbox purge #5206

Merged

benesjan force-pushed the 03-17-refactor_l1_l2_messages_cleanup branch 3 times, most recently from 049b931 to ee492ca Compare March 17, 2024 18:03

benesjan mentioned this pull request Mar 17, 2024

refactor: removing redundant receipts check #5271

Merged

benesjan force-pushed the 03-17-refactor_l1_l2_messages_cleanup branch from 1e18fc6 to aff6f38 Compare March 17, 2024 18:33

This was referenced Mar 17, 2024

refactor: nuking l1 to l2 messages from block body #5272

Merged

refactor: nuking L1 block number from L2 block #5275

Merged

benesjan marked this pull request as ready for review March 18, 2024 09:44

benesjan commented Mar 18, 2024

View reviewed changes

l1-contracts/src/core/interfaces/messagebridge/IInbox.sol Show resolved Hide resolved

benesjan commented Mar 18, 2024

View reviewed changes

l1-contracts/test/portals/TokenPortal.t.sol Show resolved Hide resolved

benesjan requested a review from LHerskind March 18, 2024 09:49

benesjan mentioned this pull request Mar 18, 2024

feat: returning non-nullified messages only #5278

Closed

benesjan force-pushed the 03-14-refactor_old_inbox_purge branch from 29497fd to 5388dce Compare March 18, 2024 10:45

benesjan force-pushed the 03-17-refactor_l1_l2_messages_cleanup branch from aff6f38 to 98a433b Compare March 18, 2024 10:45

LHerskind reviewed Mar 18, 2024

View reviewed changes

yarn-project/circuit-types/src/interfaces/block_number.ts Outdated Show resolved Hide resolved

yarn-project/circuit-types/src/messaging/inbox_leaf.ts Outdated Show resolved Hide resolved

yarn-project/circuit-types/src/messaging/l1_actor.ts Outdated Show resolved Hide resolved

benesjan force-pushed the 03-17-refactor_l1_l2_messages_cleanup branch 2 times, most recently from 047f11c to ecae75d Compare March 18, 2024 11:28

Base automatically changed from 03-14-refactor_old_inbox_purge to master March 18, 2024 11:49

benesjan force-pushed the 03-17-refactor_l1_l2_messages_cleanup branch from ecae75d to 93ac685 Compare March 18, 2024 11:56

benesjan changed the base branch from master to 03-17-refactor_removing_redundant_receipts_check March 18, 2024 11:57

benesjan force-pushed the 03-17-refactor_l1_l2_messages_cleanup branch from 93ac685 to e5157b3 Compare March 18, 2024 11:57

Base automatically changed from 03-17-refactor_removing_redundant_receipts_check to master March 18, 2024 12:32

benesjan force-pushed the 03-17-refactor_l1_l2_messages_cleanup branch 6 times, most recently from df3f298 to 4d3722f Compare March 19, 2024 07:52

benesjan added 13 commits March 19, 2024 10:30

refactor: l1 l2 messages cleanup

6cc9f5a

DataRetrieval cleanup

4607876

WIP

d9fd5c8

WIP

17518eb

WIP

c2a3161

trying to re-trigger messed up CI

0528d33

BlockNumber -> L2BlockNumber

dfd769a

comment fixes

ed27e71

bug fix

cd9deb0

bumping timeout

3d9914d

incorrect search block fix

588a1ad

Revert "bumping timeout"

e27707f

This reverts commit 4d3722f.

incorrect search block fix 2

e779f92

benesjan force-pushed the 03-17-refactor_l1_l2_messages_cleanup branch from 5fc9687 to e779f92 Compare March 19, 2024 10:36

benesjan commented Mar 19, 2024

View reviewed changes

benesjan requested a review from LHerskind March 19, 2024 11:01

LHerskind approved these changes Mar 19, 2024

View reviewed changes

benesjan merged commit 30908eb into master Mar 19, 2024

benesjan deleted the 03-17-refactor_l1_l2_messages_cleanup branch March 19, 2024 11:13

AztecBot mentioned this pull request Mar 19, 2024

chore(master): Release 0.30.0 #5296

Merged

Conversation

benesjan commented Mar 17, 2024

Uh oh!

benesjan commented Mar 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LHerskind left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

benesjan Mar 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benesjan Mar 19, 2024

Choose a reason for hiding this comment

Uh oh!

benesjan Mar 19, 2024

Choose a reason for hiding this comment

Uh oh!

benesjan Mar 19, 2024

Choose a reason for hiding this comment

Uh oh!

AztecBot commented Mar 19, 2024

Benchmark results

L2 block published to L1

L2 chain processing

Circuits stats

Tree insertion stats

Miscellaneous

Uh oh!

github-actions bot commented Mar 19, 2024

Changes to circuit sizes

🧾 Summary (100% most significant diffs)

Uh oh!

LHerskind left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

benesjan commented Mar 17, 2024 •

edited

Loading

benesjan Mar 19, 2024 •

edited

Loading