Skip to content

Implement countNotSeenAttester for OpPool - #4038

Closed
twoeths wants to merge 1 commit into
dapplion/oppool-aggregate-bitarray2from
tuyen/oppool-aggregate-bitarray2
Closed

Implement countNotSeenAttester for OpPool#4038
twoeths wants to merge 1 commit into
dapplion/oppool-aggregate-bitarray2from
tuyen/oppool-aggregate-bitarray2

Conversation

@twoeths

@twoeths twoeths commented May 19, 2022

Copy link
Copy Markdown
Member

Motivation

Improve performance of aggregated op pool, started from #4034

Description

  • Implement countNotSeenAttester given committee, attesting indices, seen attesting indices in the same order
  • Clean up some code

@codecov

codecov Bot commented May 19, 2022 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #4038 (6e0fdd4) into master (01e6ffe) will decrease coverage by 0.10%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4038      +/-   ##
==========================================
- Coverage   36.78%   36.67%   -0.11%     
==========================================
  Files         325      326       +1     
  Lines        9206     9258      +52     
  Branches     1501     1512      +11     
==========================================
+ Hits         3386     3395       +9     
- Misses       5629     5672      +43     
  Partials      191      191

@github-actions

github-actions Bot commented May 19, 2022

Copy link
Copy Markdown
Contributor

Performance Report

鉁旓笍 no performance regression detected

Full benchmark results
Benchmark suite Current: 003de4c Previous: b020ccb Ratio
BeaconState.hashTreeRoot - No change 522.00 ns/op 520.00 ns/op 1.00
BeaconState.hashTreeRoot - 1 full validator 66.409 us/op 57.041 us/op 1.16
BeaconState.hashTreeRoot - 32 full validator 653.84 us/op 559.22 us/op 1.17
BeaconState.hashTreeRoot - 512 full validator 6.9790 ms/op 6.8127 ms/op 1.02
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 83.262 us/op 81.325 us/op 1.02
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1172 ms/op 1.0076 ms/op 1.11
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 14.911 ms/op 13.848 ms/op 1.08
BeaconState.hashTreeRoot - 1 balances 63.675 us/op 55.676 us/op 1.14
BeaconState.hashTreeRoot - 32 balances 567.24 us/op 497.32 us/op 1.14
BeaconState.hashTreeRoot - 512 balances 5.4296 ms/op 4.8045 ms/op 1.13
BeaconState.hashTreeRoot - 250000 balances 110.36 ms/op 111.08 ms/op 0.99
processSlot - 1 slots 12.200 us/op 11.245 us/op 1.08
processSlot - 32 slots 1.8354 ms/op 1.6593 ms/op 1.11
getCommitteeAssignments - req 1 vs - 250000 vc 5.3010 ms/op 5.5652 ms/op 0.95
getCommitteeAssignments - req 100 vs - 250000 vc 7.8896 ms/op 8.0027 ms/op 0.99
getCommitteeAssignments - req 1000 vs - 250000 vc 8.3581 ms/op 9.2046 ms/op 0.91
computeProposers - vc 250000 17.994 ms/op 17.600 ms/op 1.02
computeEpochShuffling - vc 250000 170.60 ms/op 172.36 ms/op 0.99
getNextSyncCommittee - vc 250000 294.83 ms/op 287.09 ms/op 1.03
altair processAttestation - 250000 vs - 7PWei normalcase 3.3074 ms/op 3.9931 ms/op 0.83
altair processAttestation - 250000 vs - 7PWei worstcase 4.9548 ms/op 5.6441 ms/op 0.88
altair processAttestation - setStatus - 1/6 committees join 162.08 us/op 210.64 us/op 0.77
altair processAttestation - setStatus - 1/3 committees join 322.96 us/op 410.17 us/op 0.79
altair processAttestation - setStatus - 1/2 committees join 463.41 us/op 566.68 us/op 0.82
altair processAttestation - setStatus - 2/3 committees join 608.46 us/op 768.95 us/op 0.79
altair processAttestation - setStatus - 4/5 committees join 841.79 us/op 1.0512 ms/op 0.80
altair processAttestation - setStatus - 100% committees join 1.0253 ms/op 1.2739 ms/op 0.80
altair processBlock - 250000 vs - 7PWei normalcase 22.103 ms/op 27.692 ms/op 0.80
altair processBlock - 250000 vs - 7PWei normalcase hashState 34.523 ms/op 39.363 ms/op 0.88
altair processBlock - 250000 vs - 7PWei worstcase 75.007 ms/op 84.636 ms/op 0.89
altair processBlock - 250000 vs - 7PWei worstcase hashState 109.08 ms/op 107.90 ms/op 1.01
altair processEth1Data - 250000 vs - 7PWei normalcase 689.39 us/op 925.03 us/op 0.75
altair processEpoch - mainnet_e81889 533.29 ms/op 613.03 ms/op 0.87
mainnet_e81889 - altair beforeProcessEpoch 139.64 ms/op 172.79 ms/op 0.81
mainnet_e81889 - altair processJustificationAndFinalization 18.912 us/op 33.537 us/op 0.56
mainnet_e81889 - altair processInactivityUpdates 9.1236 ms/op 11.803 ms/op 0.77
mainnet_e81889 - altair processRewardsAndPenalties 106.78 ms/op 91.904 ms/op 1.16
mainnet_e81889 - altair processRegistryUpdates 2.9570 us/op 13.817 us/op 0.21
mainnet_e81889 - altair processSlashings 836.00 ns/op 4.3220 us/op 0.19
mainnet_e81889 - altair processEth1DataReset 848.00 ns/op 3.2960 us/op 0.26
mainnet_e81889 - altair processEffectiveBalanceUpdates 6.0114 ms/op 7.2138 ms/op 0.83
mainnet_e81889 - altair processSlashingsReset 3.8600 us/op 15.649 us/op 0.25
mainnet_e81889 - altair processRandaoMixesReset 4.9410 us/op 16.361 us/op 0.30
mainnet_e81889 - altair processHistoricalRootsUpdate 990.00 ns/op 5.1100 us/op 0.19
mainnet_e81889 - altair processParticipationFlagUpdates 3.2130 us/op 12.133 us/op 0.26
mainnet_e81889 - altair processSyncCommitteeUpdates 1.9130 us/op 3.5300 us/op 0.54
mainnet_e81889 - altair afterProcessEpoch 216.15 ms/op 200.22 ms/op 1.08
altair processInactivityUpdates - 250000 normalcase 28.072 ms/op 42.551 ms/op 0.66
altair processInactivityUpdates - 250000 worstcase 22.678 ms/op 29.037 ms/op 0.78
altair processRewardsAndPenalties - 250000 normalcase 88.709 ms/op 89.490 ms/op 0.99
altair processRewardsAndPenalties - 250000 worstcase 87.593 ms/op 142.44 ms/op 0.61
altair processSyncCommitteeUpdates - 250000 296.95 ms/op 289.10 ms/op 1.03
Tree 40 250000 create 664.23 ms/op 757.37 ms/op 0.88
Tree 40 250000 get(125000) 253.58 ns/op 313.65 ns/op 0.81
Tree 40 250000 set(125000) 1.9118 us/op 2.5860 us/op 0.74
Tree 40 250000 toArray() 25.545 ms/op 34.160 ms/op 0.75
Tree 40 250000 iterate all - toArray() + loop 25.324 ms/op 33.696 ms/op 0.75
Tree 40 250000 iterate all - get(i) 106.13 ms/op 114.70 ms/op 0.93
MutableVector 250000 create 12.392 ms/op 17.051 ms/op 0.73
MutableVector 250000 get(125000) 10.669 ns/op 15.694 ns/op 0.68
MutableVector 250000 set(125000) 530.72 ns/op 701.79 ns/op 0.76
MutableVector 250000 toArray() 5.0281 ms/op 8.1698 ms/op 0.62
MutableVector 250000 iterate all - toArray() + loop 5.1331 ms/op 7.6360 ms/op 0.67
MutableVector 250000 iterate all - get(i) 2.6014 ms/op 3.6042 ms/op 0.72
Array 250000 create 4.9819 ms/op 6.2961 ms/op 0.79
Array 250000 clone - spread 2.9542 ms/op 3.7868 ms/op 0.78
Array 250000 get(125000) 1.4850 ns/op 1.6160 ns/op 0.92
Array 250000 set(125000) 1.4800 ns/op 1.5940 ns/op 0.93
Array 250000 iterate all - loop 157.50 us/op 185.07 us/op 0.85
effectiveBalanceIncrements clone Uint8Array 300000 44.545 us/op 92.791 us/op 0.48
effectiveBalanceIncrements clone MutableVector 300000 1.0210 us/op 1.2640 us/op 0.81
effectiveBalanceIncrements rw all Uint8Array 300000 181.26 us/op 344.93 us/op 0.53
effectiveBalanceIncrements rw all MutableVector 300000 172.74 ms/op 228.55 ms/op 0.76
aggregationBits - 2048 els - zipIndexesInBitList 26.581 us/op 35.431 us/op 0.75
regular array get 100000 times 61.922 us/op 73.370 us/op 0.84
wrappedArray get 100000 times 60.619 us/op 75.081 us/op 0.81
arrayWithProxy get 100000 times 29.550 ms/op 32.812 ms/op 0.90
ssz.Root.equals 453.00 ns/op 526.00 ns/op 0.86
byteArrayEquals 434.00 ns/op 516.00 ns/op 0.84
phase0 processBlock - 250000 vs - 7PWei normalcase 3.2103 ms/op 4.5098 ms/op 0.71
phase0 processBlock - 250000 vs - 7PWei worstcase 50.984 ms/op 48.607 ms/op 1.05
phase0 afterProcessEpoch - 250000 vs - 7PWei 197.49 ms/op 189.34 ms/op 1.04
phase0 beforeProcessEpoch - 250000 vs - 7PWei 59.188 ms/op 82.494 ms/op 0.72
phase0 processEpoch - mainnet_e58758 531.07 ms/op 563.91 ms/op 0.94
mainnet_e58758 - phase0 beforeProcessEpoch 197.91 ms/op 245.77 ms/op 0.81
mainnet_e58758 - phase0 processJustificationAndFinalization 17.372 us/op 38.422 us/op 0.45
mainnet_e58758 - phase0 processRewardsAndPenalties 67.446 ms/op 93.605 ms/op 0.72
mainnet_e58758 - phase0 processRegistryUpdates 7.8730 us/op 21.019 us/op 0.37
mainnet_e58758 - phase0 processSlashings 884.00 ns/op 3.6350 us/op 0.24
mainnet_e58758 - phase0 processEth1DataReset 878.00 ns/op 3.3470 us/op 0.26
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 4.7906 ms/op 5.9018 ms/op 0.81
mainnet_e58758 - phase0 processSlashingsReset 4.6930 us/op 12.926 us/op 0.36
mainnet_e58758 - phase0 processRandaoMixesReset 4.8360 us/op 16.992 us/op 0.28
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.0350 us/op 4.8160 us/op 0.21
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.0260 us/op 12.474 us/op 0.32
mainnet_e58758 - phase0 afterProcessEpoch 163.82 ms/op 172.40 ms/op 0.95
phase0 processEffectiveBalanceUpdates - 250000 normalcase 5.4663 ms/op 7.0621 ms/op 0.77
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 6.3135 ms/op 7.6834 ms/op 0.82
phase0 processRegistryUpdates - 250000 normalcase 6.5830 us/op 20.978 us/op 0.31
phase0 processRegistryUpdates - 250000 badcase_full_deposits 374.92 us/op 659.22 us/op 0.57
phase0 processRegistryUpdates - 250000 worstcase 0.5 171.44 ms/op 226.72 ms/op 0.76
phase0 getAttestationDeltas - 250000 normalcase 13.404 ms/op 14.444 ms/op 0.93
phase0 getAttestationDeltas - 250000 worstcase 13.282 ms/op 16.229 ms/op 0.82
phase0 processSlashings - 250000 worstcase 5.2497 ms/op 6.3517 ms/op 0.83
shuffle list - 16384 els 11.217 ms/op 12.218 ms/op 0.92
shuffle list - 250000 els 167.13 ms/op 165.46 ms/op 1.01
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 362.07 us/op 551.06 us/op 0.66
pass gossip attestations to forkchoice per slot 3.0977 ms/op 3.8758 ms/op 0.80
computeDeltas 3.1241 ms/op 4.0405 ms/op 0.77
computeProposerBoostScoreFromBalances 452.25 us/op 546.99 us/op 0.83
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.5317 ms/op 1.8759 ms/op 0.82
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 73.669 us/op 89.013 us/op 0.83
BLS verify - blst-native 2.1703 ms/op 2.0382 ms/op 1.06
BLS verifyMultipleSignatures 3 - blst-native 4.4771 ms/op 4.5719 ms/op 0.98
BLS verifyMultipleSignatures 8 - blst-native 9.6760 ms/op 8.9020 ms/op 1.09
BLS verifyMultipleSignatures 32 - blst-native 35.161 ms/op 31.587 ms/op 1.11
BLS aggregatePubkeys 32 - blst-native 46.482 us/op 43.155 us/op 1.08
BLS aggregatePubkeys 128 - blst-native 182.26 us/op 163.35 us/op 1.12
getAttestationsForBlock 35.453 ms/op 71.065 ms/op 0.50
CheckpointStateCache - add get delete 10.929 us/op 12.388 us/op 0.88
validate gossip signedAggregateAndProof - struct 5.0109 ms/op 4.7016 ms/op 1.07
validate gossip attestation - struct 2.3672 ms/op 2.3457 ms/op 1.01
altair verifyImport mainnet_s3766816:31 6.8527 s/op 6.7936 s/op 1.01
pickEth1Vote - no votes 2.2428 ms/op 2.4333 ms/op 0.92
pickEth1Vote - max votes 21.039 ms/op 25.587 ms/op 0.82
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.339 ms/op 13.822 ms/op 0.89
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.047 ms/op 24.776 ms/op 0.85
pickEth1Vote - Eth1Data fastSerialize value x2048 1.5606 ms/op 2.0202 ms/op 0.77
pickEth1Vote - Eth1Data fastSerialize tree x2048 14.800 ms/op 18.293 ms/op 0.81
bytes32 toHexString 966.00 ns/op 1.0790 us/op 0.90
bytes32 Buffer.toString(hex) 748.00 ns/op 715.00 ns/op 1.05
bytes32 Buffer.toString(hex) from Uint8Array 992.00 ns/op 897.00 ns/op 1.11
bytes32 Buffer.toString(hex) + 0x 754.00 ns/op 715.00 ns/op 1.05
Object access 1 prop 0.41800 ns/op 0.36700 ns/op 1.14
Map access 1 prop 0.37600 ns/op 0.30900 ns/op 1.22
Object get x1000 10.975 ns/op 18.107 ns/op 0.61
Map get x1000 1.0370 ns/op 1.0670 ns/op 0.97
Object set x1000 70.432 ns/op 119.81 ns/op 0.59
Map set x1000 46.631 ns/op 75.029 ns/op 0.62
Return object 10000 times 0.44890 ns/op 0.41060 ns/op 1.09
Throw Error 10000 times 6.0156 us/op 6.2273 us/op 0.97
enrSubnets - fastDeserialize 64 bits 2.7800 us/op 2.8930 us/op 0.96
enrSubnets - ssz BitVector 64 bits 857.00 ns/op 785.00 ns/op 1.09
enrSubnets - fastDeserialize 4 bits 437.00 ns/op 430.00 ns/op 1.02
enrSubnets - ssz BitVector 4 bits 857.00 ns/op 801.00 ns/op 1.07
prioritizePeers score -10:0 att 32-0.1 sync 2-0 82.218 us/op 100.61 us/op 0.82
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 118.72 us/op 139.84 us/op 0.85
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 193.10 us/op 214.84 us/op 0.90
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 381.39 us/op 459.95 us/op 0.83
prioritizePeers score 0:0 att 64-1 sync 4-1 430.62 us/op 474.20 us/op 0.91
RateTracker 1000000 limit, 1 obj count per request 178.88 ns/op 198.17 ns/op 0.90
RateTracker 1000000 limit, 2 obj count per request 129.76 ns/op 143.69 ns/op 0.90
RateTracker 1000000 limit, 4 obj count per request 107.41 ns/op 114.48 ns/op 0.94
RateTracker 1000000 limit, 8 obj count per request 98.388 ns/op 108.98 ns/op 0.90
RateTracker with prune 3.9620 us/op 4.7590 us/op 0.83
array of 16000 items push then shift 51.569 us/op 3.4547 us/op 14.93
LinkedList of 16000 items push then shift 15.324 ns/op 24.209 ns/op 0.63
array of 16000 items push then pop 195.99 ns/op 241.59 ns/op 0.81
LinkedList of 16000 items push then pop 13.808 ns/op 21.982 ns/op 0.63
array of 24000 items push then shift 77.322 us/op 5.2682 us/op 14.68
LinkedList of 24000 items push then shift 18.765 ns/op 24.974 ns/op 0.75
array of 24000 items push then pop 167.97 ns/op 218.83 ns/op 0.77
LinkedList of 24000 items push then pop 15.440 ns/op 20.732 ns/op 0.74
intersect bitArray bitLen 8 9.0490 ns/op
intersect array and set length 8 134.50 ns/op
intersect bitArray bitLen 128 46.729 ns/op
intersect array and set length 128 1.7306 us/op

by benchmarkbot/action

@twoeths
twoeths changed the base branch from master to dapplion/oppool-aggregate-bitarray2 May 19, 2022 02:51
@twoeths twoeths changed the title [DNM] Oppool aggregates use BitArray only for set logic Oppool aggregates use BitArray only for set logic May 19, 2022
@twoeths

twoeths commented May 19, 2022

Copy link
Copy Markdown
Member Author

@dapplion if we pass in attestingIndices (which we already do) and seenAttestingIndices in committee order, the getAttestationsForBlock benchmark test could finish in 35ms/op

@twoeths
twoeths marked this pull request as ready for review May 19, 2022 02:56
@twoeths
twoeths requested a review from a team as a code owner May 19, 2022 02:56
@twoeths twoeths changed the title Oppool aggregates use BitArray only for set logic Implement countNotSeenAttester for OpPool May 19, 2022
@twoeths

twoeths commented May 19, 2022

Copy link
Copy Markdown
Member Author

While this PR proves a (2x) faster improvement, it's not a worth-wide improvement, will reconsider this PR if getAttestationsForBlock() is a bottle neck in the future

@twoeths twoeths closed this May 19, 2022
@twoeths
twoeths deleted the tuyen/oppool-aggregate-bitarray2 branch May 19, 2022 03:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@twoeths