Skip to content

Process gossip block immediately - #5263

Merged
twoeths merged 4 commits into
tuyen/new_gossip_queuesfrom
tuyen/process_gossip_block_immediately
Mar 28, 2023
Merged

Process gossip block immediately#5263
twoeths merged 4 commits into
tuyen/new_gossip_queuesfrom
tuyen/process_gossip_block_immediately

Conversation

@twoeths

@twoeths twoeths commented Mar 13, 2023

Copy link
Copy Markdown
Member

Motivation

  • Gossip Block needs to check bls and regen module like other gossip objects before it's processed, in the worse case the delay could be > 2s like the below test mainnet node

Screenshot 2023-03-13 at 14 20 08

  • Gossip block's signature is verified on main thread so network processor should not check for bls thread pool before processing it

Description

  • Ensure to process gossip block immediately by bypassing bls thread pool and regen check for beacon_block* topic

part of #5247

@github-actions

github-actions Bot commented Mar 13, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Performance Report

鉁旓笍 no performance regression detected

Full benchmark results
Benchmark suite Current: e37746c Previous: 15a9035 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 554.27 us/op 786.23 us/op 0.70
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 55.672 us/op 69.772 us/op 0.80
BLS verify - blst-native 1.2404 ms/op 1.5141 ms/op 0.82
BLS verifyMultipleSignatures 3 - blst-native 2.5197 ms/op 3.0276 ms/op 0.83
BLS verifyMultipleSignatures 8 - blst-native 5.4051 ms/op 6.4360 ms/op 0.84
BLS verifyMultipleSignatures 32 - blst-native 19.602 ms/op 23.438 ms/op 0.84
BLS aggregatePubkeys 32 - blst-native 26.720 us/op 32.727 us/op 0.82
BLS aggregatePubkeys 128 - blst-native 101.49 us/op 127.49 us/op 0.80
getAttestationsForBlock 62.538 ms/op 96.457 ms/op 0.65
isKnown best case - 1 super set check 267.00 ns/op 412.00 ns/op 0.65
isKnown normal case - 2 super set checks 260.00 ns/op 325.00 ns/op 0.80
isKnown worse case - 16 super set checks 259.00 ns/op 310.00 ns/op 0.84
CheckpointStateCache - add get delete 6.0400 us/op 6.8810 us/op 0.88
validate gossip signedAggregateAndProof - struct 2.8503 ms/op 3.6173 ms/op 0.79
validate gossip attestation - struct 1.3733 ms/op 1.6369 ms/op 0.84
pickEth1Vote - no votes 1.3999 ms/op 1.7113 ms/op 0.82
pickEth1Vote - max votes 12.453 ms/op 14.455 ms/op 0.86
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.9232 ms/op 12.369 ms/op 0.80
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 15.676 ms/op 20.970 ms/op 0.75
pickEth1Vote - Eth1Data fastSerialize value x2048 789.46 us/op 896.24 us/op 0.88
pickEth1Vote - Eth1Data fastSerialize tree x2048 6.0605 ms/op 8.5621 ms/op 0.71
bytes32 toHexString 729.00 ns/op 875.00 ns/op 0.83
bytes32 Buffer.toString(hex) 443.00 ns/op 544.00 ns/op 0.81
bytes32 Buffer.toString(hex) from Uint8Array 642.00 ns/op 804.00 ns/op 0.80
bytes32 Buffer.toString(hex) + 0x 452.00 ns/op 540.00 ns/op 0.84
Object access 1 prop 0.19900 ns/op 0.23900 ns/op 0.83
Map access 1 prop 0.16300 ns/op 0.19600 ns/op 0.83
Object get x1000 7.0980 ns/op 9.7560 ns/op 0.73
Map get x1000 0.66400 ns/op 0.82500 ns/op 0.80
Object set x1000 79.222 ns/op 93.655 ns/op 0.85
Map set x1000 54.651 ns/op 67.701 ns/op 0.81
Return object 10000 times 0.24910 ns/op 0.35730 ns/op 0.70
Throw Error 10000 times 4.3723 us/op 5.3407 us/op 0.82
fastMsgIdFn sha256 / 200 bytes 3.6660 us/op 4.2640 us/op 0.86
fastMsgIdFn h32 xxhash / 200 bytes 331.00 ns/op 370.00 ns/op 0.89
fastMsgIdFn h64 xxhash / 200 bytes 496.00 ns/op 573.00 ns/op 0.87
fastMsgIdFn sha256 / 1000 bytes 12.065 us/op 14.438 us/op 0.84
fastMsgIdFn h32 xxhash / 1000 bytes 460.00 ns/op 536.00 ns/op 0.86
fastMsgIdFn h64 xxhash / 1000 bytes 572.00 ns/op 624.00 ns/op 0.92
fastMsgIdFn sha256 / 10000 bytes 107.53 us/op 123.47 us/op 0.87
fastMsgIdFn h32 xxhash / 10000 bytes 2.0140 us/op 2.3350 us/op 0.86
fastMsgIdFn h64 xxhash / 10000 bytes 1.4480 us/op 1.7370 us/op 0.83
enrSubnets - fastDeserialize 64 bits 1.8700 us/op 2.1690 us/op 0.86
enrSubnets - ssz BitVector 64 bits 660.00 ns/op 743.00 ns/op 0.89
enrSubnets - fastDeserialize 4 bits 204.00 ns/op 303.00 ns/op 0.67
enrSubnets - ssz BitVector 4 bits 634.00 ns/op 748.00 ns/op 0.85
prioritizePeers score -10:0 att 32-0.1 sync 2-0 126.19 us/op 172.24 us/op 0.73
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 159.83 us/op 195.33 us/op 0.82
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 201.45 us/op 270.79 us/op 0.74
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 380.26 us/op 450.33 us/op 0.84
prioritizePeers score 0:0 att 64-1 sync 4-1 463.20 us/op 533.13 us/op 0.87
array of 16000 items push then shift 1.7090 us/op 2.0507 us/op 0.83
LinkedList of 16000 items push then shift 9.4790 ns/op 11.399 ns/op 0.83
array of 16000 items push then pop 124.67 ns/op 151.83 ns/op 0.82
LinkedList of 16000 items push then pop 9.2990 ns/op 10.924 ns/op 0.85
array of 24000 items push then shift 2.4172 us/op 2.9843 us/op 0.81
LinkedList of 24000 items push then shift 9.5370 ns/op 12.233 ns/op 0.78
array of 24000 items push then pop 89.931 ns/op 108.83 ns/op 0.83
LinkedList of 24000 items push then pop 9.1340 ns/op 10.954 ns/op 0.83
intersect bitArray bitLen 8 14.248 ns/op 16.176 ns/op 0.88
intersect array and set length 8 110.34 ns/op 134.98 ns/op 0.82
intersect bitArray bitLen 128 45.001 ns/op 58.049 ns/op 0.78
intersect array and set length 128 1.3383 us/op 1.5862 us/op 0.84
Buffer.concat 32 items 2.9850 us/op 3.8910 us/op 0.77
Uint8Array.set 32 items 2.4010 us/op 2.8090 us/op 0.85
pass gossip attestations to forkchoice per slot 2.4617 ms/op 2.7248 ms/op 0.90
computeDeltas 3.0059 ms/op 3.6714 ms/op 0.82
computeProposerBoostScoreFromBalances 1.7977 ms/op 2.1276 ms/op 0.84
altair processAttestation - 250000 vs - 7PWei normalcase 2.1908 ms/op 3.9938 ms/op 0.55
altair processAttestation - 250000 vs - 7PWei worstcase 3.3589 ms/op 5.2470 ms/op 0.64
altair processAttestation - setStatus - 1/6 committees join 138.94 us/op 179.34 us/op 0.77
altair processAttestation - setStatus - 1/3 committees join 280.27 us/op 398.57 us/op 0.70
altair processAttestation - setStatus - 1/2 committees join 371.08 us/op 450.41 us/op 0.82
altair processAttestation - setStatus - 2/3 committees join 469.58 us/op 578.47 us/op 0.81
altair processAttestation - setStatus - 4/5 committees join 645.82 us/op 808.57 us/op 0.80
altair processAttestation - setStatus - 100% committees join 757.37 us/op 917.01 us/op 0.83
altair processBlock - 250000 vs - 7PWei normalcase 16.769 ms/op 24.276 ms/op 0.69
altair processBlock - 250000 vs - 7PWei normalcase hashState 26.889 ms/op 33.777 ms/op 0.80
altair processBlock - 250000 vs - 7PWei worstcase 48.434 ms/op 69.976 ms/op 0.69
altair processBlock - 250000 vs - 7PWei worstcase hashState 72.380 ms/op 89.135 ms/op 0.81
phase0 processBlock - 250000 vs - 7PWei normalcase 2.2095 ms/op 3.1415 ms/op 0.70
phase0 processBlock - 250000 vs - 7PWei worstcase 30.630 ms/op 37.818 ms/op 0.81
altair processEth1Data - 250000 vs - 7PWei normalcase 478.36 us/op 764.14 us/op 0.63
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 8.4300 us/op 14.945 us/op 0.56
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 26.828 us/op 35.768 us/op 0.75
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 10.868 us/op 16.093 us/op 0.68
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 8.5120 us/op 15.072 us/op 0.56
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 100.28 us/op 121.80 us/op 0.82
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 659.04 us/op 957.87 us/op 0.69
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 916.88 us/op 1.3409 ms/op 0.68
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 876.99 us/op 1.4323 ms/op 0.61
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.4343 ms/op 5.6513 ms/op 0.43
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.5440 ms/op 2.6128 ms/op 0.59
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 4.0604 ms/op 6.0212 ms/op 0.67
Tree 40 250000 create 334.71 ms/op 673.22 ms/op 0.50
Tree 40 250000 get(125000) 191.73 ns/op 224.16 ns/op 0.86
Tree 40 250000 set(125000) 1.0358 us/op 2.0276 us/op 0.51
Tree 40 250000 toArray() 20.181 ms/op 28.761 ms/op 0.70
Tree 40 250000 iterate all - toArray() + loop 21.163 ms/op 29.040 ms/op 0.73
Tree 40 250000 iterate all - get(i) 75.315 ms/op 92.254 ms/op 0.82
MutableVector 250000 create 11.822 ms/op 15.534 ms/op 0.76
MutableVector 250000 get(125000) 6.5180 ns/op 8.2150 ns/op 0.79
MutableVector 250000 set(125000) 297.42 ns/op 448.91 ns/op 0.66
MutableVector 250000 toArray() 3.2471 ms/op 4.3719 ms/op 0.74
MutableVector 250000 iterate all - toArray() + loop 3.4253 ms/op 4.7981 ms/op 0.71
MutableVector 250000 iterate all - get(i) 1.8912 ms/op 1.9329 ms/op 0.98
Array 250000 create 2.7336 ms/op 4.0928 ms/op 0.67
Array 250000 clone - spread 1.3296 ms/op 1.7571 ms/op 0.76
Array 250000 get(125000) 0.61500 ns/op 1.1270 ns/op 0.55
Array 250000 set(125000) 0.70800 ns/op 1.1710 ns/op 0.60
Array 250000 iterate all - loop 86.977 us/op 105.17 us/op 0.83
effectiveBalanceIncrements clone Uint8Array 300000 50.749 us/op 54.873 us/op 0.92
effectiveBalanceIncrements clone MutableVector 300000 477.00 ns/op 607.00 ns/op 0.79
effectiveBalanceIncrements rw all Uint8Array 300000 176.27 us/op 215.14 us/op 0.82
effectiveBalanceIncrements rw all MutableVector 300000 139.64 ms/op 162.26 ms/op 0.86
phase0 afterProcessEpoch - 250000 vs - 7PWei 128.67 ms/op 142.08 ms/op 0.91
phase0 beforeProcessEpoch - 250000 vs - 7PWei 38.981 ms/op 53.120 ms/op 0.73
altair processEpoch - mainnet_e81889 336.85 ms/op 411.03 ms/op 0.82
mainnet_e81889 - altair beforeProcessEpoch 73.642 ms/op 78.300 ms/op 0.94
mainnet_e81889 - altair processJustificationAndFinalization 19.104 us/op 40.498 us/op 0.47
mainnet_e81889 - altair processInactivityUpdates 5.7187 ms/op 7.4149 ms/op 0.77
mainnet_e81889 - altair processRewardsAndPenalties 71.285 ms/op 63.105 ms/op 1.13
mainnet_e81889 - altair processRegistryUpdates 3.0930 us/op 9.3600 us/op 0.33
mainnet_e81889 - altair processSlashings 654.00 ns/op 1.3800 us/op 0.47
mainnet_e81889 - altair processEth1DataReset 523.00 ns/op 1.4980 us/op 0.35
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2823 ms/op 1.6237 ms/op 0.79
mainnet_e81889 - altair processSlashingsReset 5.1910 us/op 9.9720 us/op 0.52
mainnet_e81889 - altair processRandaoMixesReset 4.6760 us/op 16.576 us/op 0.28
mainnet_e81889 - altair processHistoricalRootsUpdate 2.0200 us/op 2.3290 us/op 0.87
mainnet_e81889 - altair processParticipationFlagUpdates 3.7410 us/op 8.0780 us/op 0.46
mainnet_e81889 - altair processSyncCommitteeUpdates 789.00 ns/op 1.5780 us/op 0.50
mainnet_e81889 - altair afterProcessEpoch 130.12 ms/op 160.24 ms/op 0.81
phase0 processEpoch - mainnet_e58758 374.42 ms/op 460.68 ms/op 0.81
mainnet_e58758 - phase0 beforeProcessEpoch 134.78 ms/op 173.44 ms/op 0.78
mainnet_e58758 - phase0 processJustificationAndFinalization 20.968 us/op 30.920 us/op 0.68
mainnet_e58758 - phase0 processRewardsAndPenalties 65.926 ms/op 85.326 ms/op 0.77
mainnet_e58758 - phase0 processRegistryUpdates 8.4210 us/op 19.683 us/op 0.43
mainnet_e58758 - phase0 processSlashings 492.00 ns/op 1.6680 us/op 0.29
mainnet_e58758 - phase0 processEth1DataReset 493.00 ns/op 1.5570 us/op 0.32
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.5093 ms/op 1.9082 ms/op 0.79
mainnet_e58758 - phase0 processSlashingsReset 3.4430 us/op 10.700 us/op 0.32
mainnet_e58758 - phase0 processRandaoMixesReset 4.8380 us/op 14.635 us/op 0.33
mainnet_e58758 - phase0 processHistoricalRootsUpdate 599.00 ns/op 2.3390 us/op 0.26
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.2170 us/op 13.162 us/op 0.32
mainnet_e58758 - phase0 afterProcessEpoch 100.94 ms/op 125.46 ms/op 0.80
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2515 ms/op 1.4674 ms/op 0.85
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.5136 ms/op 1.8685 ms/op 0.81
altair processInactivityUpdates - 250000 normalcase 23.736 ms/op 38.433 ms/op 0.62
altair processInactivityUpdates - 250000 worstcase 26.922 ms/op 33.540 ms/op 0.80
phase0 processRegistryUpdates - 250000 normalcase 7.4450 us/op 17.513 us/op 0.43
phase0 processRegistryUpdates - 250000 badcase_full_deposits 267.12 us/op 460.76 us/op 0.58
phase0 processRegistryUpdates - 250000 worstcase 0.5 130.73 ms/op 189.19 ms/op 0.69
altair processRewardsAndPenalties - 250000 normalcase 64.608 ms/op 86.895 ms/op 0.74
altair processRewardsAndPenalties - 250000 worstcase 73.775 ms/op 94.281 ms/op 0.78
phase0 getAttestationDeltas - 250000 normalcase 6.6080 ms/op 10.286 ms/op 0.64
phase0 getAttestationDeltas - 250000 worstcase 7.0206 ms/op 10.931 ms/op 0.64
phase0 processSlashings - 250000 worstcase 3.4946 ms/op 5.0217 ms/op 0.70
altair processSyncCommitteeUpdates - 250000 172.35 ms/op 248.62 ms/op 0.69
BeaconState.hashTreeRoot - No change 260.00 ns/op 553.00 ns/op 0.47
BeaconState.hashTreeRoot - 1 full validator 53.498 us/op 69.871 us/op 0.77
BeaconState.hashTreeRoot - 32 full validator 526.13 us/op 724.42 us/op 0.73
BeaconState.hashTreeRoot - 512 full validator 5.3149 ms/op 8.1222 ms/op 0.65
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 61.998 us/op 83.010 us/op 0.75
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 885.75 us/op 1.3106 ms/op 0.68
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.228 ms/op 19.339 ms/op 0.58
BeaconState.hashTreeRoot - 1 balances 47.526 us/op 64.210 us/op 0.74
BeaconState.hashTreeRoot - 32 balances 441.70 us/op 589.52 us/op 0.75
BeaconState.hashTreeRoot - 512 balances 4.4034 ms/op 6.6093 ms/op 0.67
BeaconState.hashTreeRoot - 250000 balances 76.600 ms/op 99.784 ms/op 0.77
aggregationBits - 2048 els - zipIndexesInBitList 15.866 us/op 28.983 us/op 0.55
regular array get 100000 times 33.260 us/op 51.271 us/op 0.65
wrappedArray get 100000 times 33.393 us/op 42.717 us/op 0.78
arrayWithProxy get 100000 times 16.305 ms/op 21.611 ms/op 0.75
ssz.Root.equals 587.00 ns/op 746.00 ns/op 0.79
byteArrayEquals 540.00 ns/op 686.00 ns/op 0.79
shuffle list - 16384 els 6.9571 ms/op 8.5471 ms/op 0.81
shuffle list - 250000 els 99.107 ms/op 131.87 ms/op 0.75
processSlot - 1 slots 8.5140 us/op 14.063 us/op 0.61
processSlot - 32 slots 1.3229 ms/op 2.0534 ms/op 0.64
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 190.60 us/op 256.13 us/op 0.74
getCommitteeAssignments - req 1 vs - 250000 vc 2.8358 ms/op 3.5672 ms/op 0.79
getCommitteeAssignments - req 100 vs - 250000 vc 4.0584 ms/op 5.1490 ms/op 0.79
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3415 ms/op 5.5448 ms/op 0.78
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.4600 ns/op 5.9100 ns/op 0.75
state getBlockRootAtSlot - 250000 vs - 7PWei 545.70 ns/op 891.75 ns/op 0.61
computeProposers - vc 250000 10.121 ms/op 17.145 ms/op 0.59
computeEpochShuffling - vc 250000 98.977 ms/op 135.28 ms/op 0.73
getNextSyncCommittee - vc 250000 169.13 ms/op 235.70 ms/op 0.72

by benchmarkbot/action

dapplion
dapplion reviewed Mar 13, 2023
View reviewed changes
Comment thread packages/beacon-node/src/network/processor/index.ts Outdated
@twoeths
twoeths marked this pull request as ready for review March 13, 2023 09:50
@twoeths
twoeths requested a review from a team as a code owner March 13, 2023 09:50
dapplion
dapplion reviewed Mar 14, 2023
View reviewed changes
Comment thread packages/beacon-node/src/network/processor/index.ts Outdated
// Check canAcceptWork before calling queue.next() since it consumes the items
// beacon block is guaranteed to be processed immedately
const bypassQueue = executeGossipWorkOrderObj[topic]?.bypassQueue ?? false;
if (!bypassQueue && (!this.chain.blsThreadPoolCanAcceptWork() || !this.chain.regenCanAcceptWork())) {

@dapplion dapplion Mar 14, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will cause to have to call canAcceptWork many more times. Why not:

const canAcceptWork = this.chain.blsThreadPoolCanAcceptWork() && this.chain.regenCanAcceptWork()

for (const topic of executeGossipWorkOrder) {
        // Check canAcceptWork before calling queue.next() since it consumes the items
        // beacon block is guaranteed to be processed immedately
        if ( !canAcceptWork && !executeGossipWorkOrderObj[topic].bypassQueue) {
           break job_loop;
        }

@twoeths twoeths Mar 14, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point 馃憤 , I modified accordingly

@twoeths
twoeths marked this pull request as draft March 15, 2023 03:54
@twoeths

twoeths commented Mar 15, 2023

Copy link
Copy Markdown
Member Author

convert to draft, new gossipsub queue work will be reverted once we merge stable v1.6.0 to unstable

dapplion
dapplion previously approved these changes Mar 19, 2023
View reviewed changes
@twoeths
twoeths force-pushed the tuyen/process_gossip_block_immediately branch from 2214686 to b1a2726 Compare March 28, 2023 03:01
@twoeths
twoeths changed the base branch from unstable to tuyen/new_gossip_queues March 28, 2023 03:01
@twoeths
twoeths marked this pull request as ready for review March 28, 2023 03:01
@twoeths
twoeths merged commit c39c6e6 into tuyen/new_gossip_queues Mar 28, 2023
@twoeths
twoeths deleted the tuyen/process_gossip_block_immediately branch March 28, 2023 03:02
@twoeths

twoeths commented Mar 28, 2023

Copy link
Copy Markdown
Member Author

Merged to tuyen/new_gossip_queues as pre-approved by @dapplion

twoeths added a commit that referenced this pull request Apr 7, 2023
@twoeths
Process gossip block immediately (#5263)
34b8b92 
* Process gossip block immediately

* Process beacon_block_and_blobs_sidecar gossip object immediately

* New WorkOpts type

* Check canAceptWork once per executeWork()
@twoeths twoeths mentioned this pull request Apr 7, 2023
Merged
wemeetagain pushed a commit that referenced this pull request Apr 7, 2023
@twoeths @wemeetagain
Process gossip block immediately (#5263)
904d86a 
* Process gossip block immediately

* Process beacon_block_and_blobs_sidecar gossip object immediately

* New WorkOpts type

* Check canAceptWork once per executeWork()
@wemeetagain

wemeetagain commented May 4, 2023

Copy link
Copy Markdown
Member

馃帀 This PR is included in v1.8.0 馃帀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@dapplion dapplion dapplion left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@twoeths @wemeetagain @dapplion