Skip to content

feat: enable deneb spec tests#5523

Merged
wemeetagain merged 4 commits intounstablefrom
ftb/enable-deneb-specs
May 24, 2023
Merged

feat: enable deneb spec tests#5523
wemeetagain merged 4 commits intounstablefrom
ftb/enable-deneb-specs

Conversation

@g11tech
Copy link
Contributor

@g11tech g11tech commented May 23, 2023

Add remaining deneb types and enable deneb spec tests

Build on top of:

@g11tech g11tech requested a review from a team as a code owner May 23, 2023 08:15
@g11tech g11tech marked this pull request as draft May 23, 2023 08:16
Base automatically changed from ftb/dbs to unstable May 24, 2023 07:57
@g11tech g11tech marked this pull request as ready for review May 24, 2023 07:58
@g11tech g11tech force-pushed the ftb/enable-deneb-specs branch from 89cbf60 to 61ad3c6 Compare May 24, 2023 08:19
@github-actions
Copy link
Contributor

github-actions bot commented May 24, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: f9ea6f9 Previous: 1f1ef67 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 896.53 us/op 783.06 us/op 1.14
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 49.274 us/op 46.924 us/op 1.05
BLS verify - blst-native 1.2411 ms/op 1.2430 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 2.4866 ms/op 2.5308 ms/op 0.98
BLS verifyMultipleSignatures 8 - blst-native 5.3043 ms/op 5.4100 ms/op 0.98
BLS verifyMultipleSignatures 32 - blst-native 19.034 ms/op 19.431 ms/op 0.98
BLS aggregatePubkeys 32 - blst-native 25.817 us/op 26.372 us/op 0.98
BLS aggregatePubkeys 128 - blst-native 101.02 us/op 103.08 us/op 0.98
getAttestationsForBlock 61.855 ms/op 63.519 ms/op 0.97
isKnown best case - 1 super set check 266.00 ns/op 252.00 ns/op 1.06
isKnown normal case - 2 super set checks 264.00 ns/op 240.00 ns/op 1.10
isKnown worse case - 16 super set checks 259.00 ns/op 242.00 ns/op 1.07
CheckpointStateCache - add get delete 5.2100 us/op 4.8290 us/op 1.08
validate gossip signedAggregateAndProof - struct 2.8352 ms/op 2.6862 ms/op 1.06
validate gossip attestation - struct 1.3411 ms/op 1.2833 ms/op 1.04
pickEth1Vote - no votes 1.4391 ms/op 1.1754 ms/op 1.22
pickEth1Vote - max votes 11.537 ms/op 12.298 ms/op 0.94
pickEth1Vote - Eth1Data hashTreeRoot value x2048 9.6596 ms/op 8.9923 ms/op 1.07
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 16.397 ms/op 18.533 ms/op 0.88
pickEth1Vote - Eth1Data fastSerialize value x2048 807.08 us/op 638.72 us/op 1.26
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.6603 ms/op 9.0738 ms/op 0.62
bytes32 toHexString 623.00 ns/op 489.00 ns/op 1.27
bytes32 Buffer.toString(hex) 401.00 ns/op 347.00 ns/op 1.16
bytes32 Buffer.toString(hex) from Uint8Array 620.00 ns/op 531.00 ns/op 1.17
bytes32 Buffer.toString(hex) + 0x 397.00 ns/op 331.00 ns/op 1.20
Object access 1 prop 0.17600 ns/op 0.15400 ns/op 1.14
Map access 1 prop 0.16700 ns/op 0.16400 ns/op 1.02
Object get x1000 6.5970 ns/op 6.5050 ns/op 1.01
Map get x1000 0.63400 ns/op 0.59400 ns/op 1.07
Object set x1000 54.475 ns/op 49.215 ns/op 1.11
Map set x1000 44.241 ns/op 41.792 ns/op 1.06
Return object 10000 times 0.24250 ns/op 0.22870 ns/op 1.06
Throw Error 10000 times 4.4201 us/op 4.0796 us/op 1.08
fastMsgIdFn sha256 / 200 bytes 3.6800 us/op 3.3650 us/op 1.09
fastMsgIdFn h32 xxhash / 200 bytes 329.00 ns/op 261.00 ns/op 1.26
fastMsgIdFn h64 xxhash / 200 bytes 481.00 ns/op 369.00 ns/op 1.30
fastMsgIdFn sha256 / 1000 bytes 12.111 us/op 11.528 us/op 1.05
fastMsgIdFn h32 xxhash / 1000 bytes 452.00 ns/op 391.00 ns/op 1.16
fastMsgIdFn h64 xxhash / 1000 bytes 514.00 ns/op 440.00 ns/op 1.17
fastMsgIdFn sha256 / 10000 bytes 104.61 us/op 101.25 us/op 1.03
fastMsgIdFn h32 xxhash / 10000 bytes 2.0380 us/op 1.8350 us/op 1.11
fastMsgIdFn h64 xxhash / 10000 bytes 1.4760 us/op 1.3010 us/op 1.13
enrSubnets - fastDeserialize 64 bits 1.4640 us/op 1.2120 us/op 1.21
enrSubnets - ssz BitVector 64 bits 583.00 ns/op 461.00 ns/op 1.26
enrSubnets - fastDeserialize 4 bits 194.00 ns/op 159.00 ns/op 1.22
enrSubnets - ssz BitVector 4 bits 568.00 ns/op 461.00 ns/op 1.23
prioritizePeers score -10:0 att 32-0.1 sync 2-0 110.42 us/op 103.73 us/op 1.06
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 149.20 us/op 129.44 us/op 1.15
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 179.66 us/op 165.62 us/op 1.08
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 377.19 us/op 296.04 us/op 1.27
prioritizePeers score 0:0 att 64-1 sync 4-1 462.41 us/op 365.98 us/op 1.26
array of 16000 items push then shift 1.6862 us/op 1.6304 us/op 1.03
LinkedList of 16000 items push then shift 9.0190 ns/op 8.9030 ns/op 1.01
array of 16000 items push then pop 112.72 ns/op 79.962 ns/op 1.41
LinkedList of 16000 items push then pop 9.0790 ns/op 8.4660 ns/op 1.07
array of 24000 items push then shift 2.4119 us/op 2.3416 us/op 1.03
LinkedList of 24000 items push then shift 9.3390 ns/op 8.7530 ns/op 1.07
array of 24000 items push then pop 87.176 ns/op 72.478 ns/op 1.20
LinkedList of 24000 items push then pop 8.9540 ns/op 8.5270 ns/op 1.05
intersect bitArray bitLen 8 13.966 ns/op 13.324 ns/op 1.05
intersect array and set length 8 90.154 ns/op 75.708 ns/op 1.19
intersect bitArray bitLen 128 46.359 ns/op 44.222 ns/op 1.05
intersect array and set length 128 1.2398 us/op 1.0431 us/op 1.19
Buffer.concat 32 items 3.7930 us/op 2.6370 us/op 1.44
Uint8Array.set 32 items 2.2700 us/op 2.1770 us/op 1.04
pass gossip attestations to forkchoice per slot 2.8994 ms/op 2.4797 ms/op 1.17
computeDeltas 3.0768 ms/op 4.2772 ms/op 0.72
computeProposerBoostScoreFromBalances 1.8063 ms/op 1.7127 ms/op 1.05
altair processAttestation - 250000 vs - 7PWei normalcase 2.7669 ms/op 2.1199 ms/op 1.31
altair processAttestation - 250000 vs - 7PWei worstcase 3.8779 ms/op 3.3192 ms/op 1.17
altair processAttestation - setStatus - 1/6 committees join 169.39 us/op 133.89 us/op 1.27
altair processAttestation - setStatus - 1/3 committees join 287.11 us/op 274.23 us/op 1.05
altair processAttestation - setStatus - 1/2 committees join 386.02 us/op 369.97 us/op 1.04
altair processAttestation - setStatus - 2/3 committees join 477.62 us/op 462.58 us/op 1.03
altair processAttestation - setStatus - 4/5 committees join 695.08 us/op 641.74 us/op 1.08
altair processAttestation - setStatus - 100% committees join 809.79 us/op 820.00 us/op 0.99
altair processBlock - 250000 vs - 7PWei normalcase 24.226 ms/op 19.834 ms/op 1.22
altair processBlock - 250000 vs - 7PWei normalcase hashState 32.366 ms/op 29.278 ms/op 1.11
altair processBlock - 250000 vs - 7PWei worstcase 69.032 ms/op 67.141 ms/op 1.03
altair processBlock - 250000 vs - 7PWei worstcase hashState 87.609 ms/op 87.427 ms/op 1.00
phase0 processBlock - 250000 vs - 7PWei normalcase 3.2682 ms/op 2.1045 ms/op 1.55
phase0 processBlock - 250000 vs - 7PWei worstcase 33.461 ms/op 29.092 ms/op 1.15
altair processEth1Data - 250000 vs - 7PWei normalcase 632.18 us/op 463.45 us/op 1.36
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 9.6700 us/op 7.6100 us/op 1.27
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 35.854 us/op 27.579 us/op 1.30
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 13.301 us/op 9.9270 us/op 1.34
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 13.483 us/op 8.7250 us/op 1.55
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 143.41 us/op 98.775 us/op 1.45
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 932.21 us/op 1.2518 ms/op 0.74
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 1.1539 ms/op 905.90 us/op 1.27
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 954.23 us/op 1.4250 ms/op 0.67
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 3.3870 ms/op 2.3893 ms/op 1.42
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.9065 ms/op 1.9380 ms/op 0.98
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 6.8520 ms/op 4.1810 ms/op 1.64
Tree 40 250000 create 480.74 ms/op 320.19 ms/op 1.50
Tree 40 250000 get(125000) 202.63 ns/op 194.41 ns/op 1.04
Tree 40 250000 set(125000) 1.4831 us/op 990.25 ns/op 1.50
Tree 40 250000 toArray() 24.385 ms/op 21.596 ms/op 1.13
Tree 40 250000 iterate all - toArray() + loop 24.281 ms/op 22.538 ms/op 1.08
Tree 40 250000 iterate all - get(i) 76.378 ms/op 75.935 ms/op 1.01
MutableVector 250000 create 12.061 ms/op 13.485 ms/op 0.89
MutableVector 250000 get(125000) 6.4680 ns/op 6.5230 ns/op 0.99
MutableVector 250000 set(125000) 319.33 ns/op 269.33 ns/op 1.19
MutableVector 250000 toArray() 4.0005 ms/op 3.5872 ms/op 1.12
MutableVector 250000 iterate all - toArray() + loop 3.9715 ms/op 3.7131 ms/op 1.07
MutableVector 250000 iterate all - get(i) 1.6260 ms/op 1.5452 ms/op 1.05
Array 250000 create 3.5347 ms/op 3.4010 ms/op 1.04
Array 250000 clone - spread 1.2261 ms/op 1.3343 ms/op 0.92
Array 250000 get(125000) 0.60800 ns/op 0.61600 ns/op 0.99
Array 250000 set(125000) 0.68900 ns/op 0.69700 ns/op 0.99
Array 250000 iterate all - loop 85.880 us/op 103.95 us/op 0.83
effectiveBalanceIncrements clone Uint8Array 300000 41.179 us/op 38.888 us/op 1.06
effectiveBalanceIncrements clone MutableVector 300000 376.00 ns/op 376.00 ns/op 1.00
effectiveBalanceIncrements rw all Uint8Array 300000 172.12 us/op 168.81 us/op 1.02
effectiveBalanceIncrements rw all MutableVector 300000 108.65 ms/op 87.448 ms/op 1.24
phase0 afterProcessEpoch - 250000 vs - 7PWei 131.69 ms/op 118.63 ms/op 1.11
phase0 beforeProcessEpoch - 250000 vs - 7PWei 65.505 ms/op 35.178 ms/op 1.86
altair processEpoch - mainnet_e81889 358.65 ms/op 345.50 ms/op 1.04
mainnet_e81889 - altair beforeProcessEpoch 88.174 ms/op 68.737 ms/op 1.28
mainnet_e81889 - altair processJustificationAndFinalization 28.738 us/op 16.849 us/op 1.71
mainnet_e81889 - altair processInactivityUpdates 7.2862 ms/op 6.0102 ms/op 1.21
mainnet_e81889 - altair processRewardsAndPenalties 79.485 ms/op 71.628 ms/op 1.11
mainnet_e81889 - altair processRegistryUpdates 7.6840 us/op 2.6700 us/op 2.88
mainnet_e81889 - altair processSlashings 1.0990 us/op 489.00 ns/op 2.25
mainnet_e81889 - altair processEth1DataReset 1.5020 us/op 531.00 ns/op 2.83
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.1636 ms/op 1.5147 ms/op 1.43
mainnet_e81889 - altair processSlashingsReset 5.8950 us/op 6.3130 us/op 0.93
mainnet_e81889 - altair processRandaoMixesReset 13.509 us/op 6.9580 us/op 1.94
mainnet_e81889 - altair processHistoricalRootsUpdate 1.5490 us/op 753.00 ns/op 2.06
mainnet_e81889 - altair processParticipationFlagUpdates 4.2150 us/op 3.8030 us/op 1.11
mainnet_e81889 - altair processSyncCommitteeUpdates 1.3020 us/op 1.3550 us/op 0.96
mainnet_e81889 - altair afterProcessEpoch 144.75 ms/op 134.17 ms/op 1.08
phase0 processEpoch - mainnet_e58758 473.81 ms/op 397.68 ms/op 1.19
mainnet_e58758 - phase0 beforeProcessEpoch 150.17 ms/op 168.89 ms/op 0.89
mainnet_e58758 - phase0 processJustificationAndFinalization 27.489 us/op 23.555 us/op 1.17
mainnet_e58758 - phase0 processRewardsAndPenalties 78.312 ms/op 70.696 ms/op 1.11
mainnet_e58758 - phase0 processRegistryUpdates 13.671 us/op 12.866 us/op 1.06
mainnet_e58758 - phase0 processSlashings 1.1290 us/op 1.1440 us/op 0.99
mainnet_e58758 - phase0 processEth1DataReset 967.00 ns/op 849.00 ns/op 1.14
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.5479 ms/op 1.2715 ms/op 1.22
mainnet_e58758 - phase0 processSlashingsReset 7.7370 us/op 4.7250 us/op 1.64
mainnet_e58758 - phase0 processRandaoMixesReset 8.4590 us/op 7.6390 us/op 1.11
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.1710 us/op 754.00 ns/op 1.55
mainnet_e58758 - phase0 processParticipationRecordUpdates 10.031 us/op 5.9920 us/op 1.67
mainnet_e58758 - phase0 afterProcessEpoch 114.94 ms/op 103.93 ms/op 1.11
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.1497 ms/op 1.4635 ms/op 1.47
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.2506 ms/op 1.9168 ms/op 1.70
altair processInactivityUpdates - 250000 normalcase 37.586 ms/op 28.454 ms/op 1.32
altair processInactivityUpdates - 250000 worstcase 39.319 ms/op 31.829 ms/op 1.24
phase0 processRegistryUpdates - 250000 normalcase 11.967 us/op 8.0130 us/op 1.49
phase0 processRegistryUpdates - 250000 badcase_full_deposits 311.80 us/op 359.61 us/op 0.87
phase0 processRegistryUpdates - 250000 worstcase 0.5 179.22 ms/op 145.90 ms/op 1.23
altair processRewardsAndPenalties - 250000 normalcase 83.973 ms/op 68.661 ms/op 1.22
altair processRewardsAndPenalties - 250000 worstcase 93.887 ms/op 74.046 ms/op 1.27
phase0 getAttestationDeltas - 250000 normalcase 11.627 ms/op 7.1711 ms/op 1.62
phase0 getAttestationDeltas - 250000 worstcase 10.647 ms/op 6.9701 ms/op 1.53
phase0 processSlashings - 250000 worstcase 3.8042 ms/op 3.4846 ms/op 1.09
altair processSyncCommitteeUpdates - 250000 199.22 ms/op 178.23 ms/op 1.12
BeaconState.hashTreeRoot - No change 363.00 ns/op 318.00 ns/op 1.14
BeaconState.hashTreeRoot - 1 full validator 56.925 us/op 61.951 us/op 0.92
BeaconState.hashTreeRoot - 32 full validator 620.21 us/op 610.57 us/op 1.02
BeaconState.hashTreeRoot - 512 full validator 6.0903 ms/op 6.2685 ms/op 0.97
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 66.326 us/op 66.275 us/op 1.00
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.0352 ms/op 995.49 us/op 1.04
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.560 ms/op 11.799 ms/op 1.40
BeaconState.hashTreeRoot - 1 balances 60.115 us/op 59.134 us/op 1.02
BeaconState.hashTreeRoot - 32 balances 569.98 us/op 467.27 us/op 1.22
BeaconState.hashTreeRoot - 512 balances 5.8383 ms/op 4.9145 ms/op 1.19
BeaconState.hashTreeRoot - 250000 balances 97.186 ms/op 81.631 ms/op 1.19
aggregationBits - 2048 els - zipIndexesInBitList 38.505 us/op 20.444 us/op 1.88
regular array get 100000 times 37.341 us/op 43.764 us/op 0.85
wrappedArray get 100000 times 37.453 us/op 35.248 us/op 1.06
arrayWithProxy get 100000 times 20.591 ms/op 18.526 ms/op 1.11
ssz.Root.equals 963.00 ns/op 625.00 ns/op 1.54
byteArrayEquals 781.00 ns/op 656.00 ns/op 1.19
shuffle list - 16384 els 8.7587 ms/op 7.7671 ms/op 1.13
shuffle list - 250000 els 113.22 ms/op 108.86 ms/op 1.04
processSlot - 1 slots 10.348 us/op 11.272 us/op 0.92
processSlot - 32 slots 1.7276 ms/op 1.4598 ms/op 1.18
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 44.955 ms/op 40.755 ms/op 1.10
getCommitteeAssignments - req 1 vs - 250000 vc 3.1168 ms/op 3.0609 ms/op 1.02
getCommitteeAssignments - req 100 vs - 250000 vc 4.4002 ms/op 4.5935 ms/op 0.96
getCommitteeAssignments - req 1000 vs - 250000 vc 4.8928 ms/op 5.5271 ms/op 0.89
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.5300 ns/op 5.2000 ns/op 1.06
state getBlockRootAtSlot - 250000 vs - 7PWei 827.41 ns/op 720.49 ns/op 1.15
computeProposers - vc 250000 12.562 ms/op 11.269 ms/op 1.11
computeEpochShuffling - vc 250000 122.96 ms/op 108.76 ms/op 1.13
getNextSyncCommittee - vc 250000 181.89 ms/op 181.66 ms/op 1.00
computeSigningRoot for AttestationData 13.404 us/op 14.472 us/op 0.93
hash AttestationData serialized data then Buffer.toString(base64) 2.5225 us/op 2.5857 us/op 0.98
toHexString serialized data 1.2137 us/op 1.3697 us/op 0.89
Buffer.toString(base64) 327.44 ns/op 409.52 ns/op 0.80

by benchmarkbot/action

@wemeetagain wemeetagain merged commit 5573bbc into unstable May 24, 2023
@wemeetagain wemeetagain deleted the ftb/enable-deneb-specs branch May 24, 2023 15:57
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants