-
-
Notifications
You must be signed in to change notification settings - Fork 289
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Disable workers for e2e test runs #4325
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Benchmark suite | Current: 8aa3f51 | Previous: fdd4c6b | Ratio |
---|---|---|---|
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 1.4798 ms/op | 382.59 us/op | 3.87 |
Full benchmark results
Benchmark suite | Current: 8aa3f51 | Previous: fdd4c6b | Ratio |
---|---|---|---|
getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 2.5328 ms/op | 1.7909 ms/op | 1.41 |
getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 83.936 us/op | 64.738 us/op | 1.30 |
BLS verify - blst-native | 2.2237 ms/op | 2.1673 ms/op | 1.03 |
BLS verifyMultipleSignatures 3 - blst-native | 4.5095 ms/op | 4.4711 ms/op | 1.01 |
BLS verifyMultipleSignatures 8 - blst-native | 9.6149 ms/op | 9.6702 ms/op | 0.99 |
BLS verifyMultipleSignatures 32 - blst-native | 34.779 ms/op | 35.132 ms/op | 0.99 |
BLS aggregatePubkeys 32 - blst-native | 46.579 us/op | 46.815 us/op | 0.99 |
BLS aggregatePubkeys 128 - blst-native | 182.54 us/op | 182.70 us/op | 1.00 |
getAttestationsForBlock | 55.307 ms/op | 41.850 ms/op | 1.32 |
isKnown best case - 1 super set check | 514.00 ns/op | 488.00 ns/op | 1.05 |
isKnown normal case - 2 super set checks | 503.00 ns/op | 475.00 ns/op | 1.06 |
isKnown worse case - 16 super set checks | 497.00 ns/op | 478.00 ns/op | 1.04 |
CheckpointStateCache - add get delete | 10.469 us/op | 9.2340 us/op | 1.13 |
validate gossip signedAggregateAndProof - struct | 5.0806 ms/op | 5.0270 ms/op | 1.01 |
validate gossip attestation - struct | 2.3838 ms/op | 2.3751 ms/op | 1.00 |
altair verifyImport mainnet_s3766816:31 | 13.926 s/op | 12.664 s/op | 1.10 |
pickEth1Vote - no votes | 2.4761 ms/op | 2.1150 ms/op | 1.17 |
pickEth1Vote - max votes | 27.452 ms/op | 20.230 ms/op | 1.36 |
pickEth1Vote - Eth1Data hashTreeRoot value x2048 | 13.621 ms/op | 13.317 ms/op | 1.02 |
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 | 25.676 ms/op | 20.891 ms/op | 1.23 |
pickEth1Vote - Eth1Data fastSerialize value x2048 | 1.8844 ms/op | 1.4652 ms/op | 1.29 |
pickEth1Vote - Eth1Data fastSerialize tree x2048 | 18.839 ms/op | 14.605 ms/op | 1.29 |
bytes32 toHexString | 1.3450 us/op | 1.0530 us/op | 1.28 |
bytes32 Buffer.toString(hex) | 889.00 ns/op | 788.00 ns/op | 1.13 |
bytes32 Buffer.toString(hex) from Uint8Array | 1.1150 us/op | 1.0140 us/op | 1.10 |
bytes32 Buffer.toString(hex) + 0x | 899.00 ns/op | 798.00 ns/op | 1.13 |
Object access 1 prop | 0.46900 ns/op | 0.38200 ns/op | 1.23 |
Map access 1 prop | 0.33400 ns/op | 0.30300 ns/op | 1.10 |
Object get x1000 | 20.982 ns/op | 11.373 ns/op | 1.84 |
Map get x1000 | 1.1460 ns/op | 0.95500 ns/op | 1.20 |
Object set x1000 | 145.41 ns/op | 75.630 ns/op | 1.92 |
Map set x1000 | 90.858 ns/op | 49.283 ns/op | 1.84 |
Return object 10000 times | 0.44570 ns/op | 0.44360 ns/op | 1.00 |
Throw Error 10000 times | 7.0810 us/op | 6.0727 us/op | 1.17 |
enrSubnets - fastDeserialize 64 bits | 3.4160 us/op | 2.8210 us/op | 1.21 |
enrSubnets - ssz BitVector 64 bits | 878.00 ns/op | 789.00 ns/op | 1.11 |
enrSubnets - fastDeserialize 4 bits | 485.00 ns/op | 393.00 ns/op | 1.23 |
enrSubnets - ssz BitVector 4 bits | 863.00 ns/op | 753.00 ns/op | 1.15 |
prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 117.28 us/op | 87.711 us/op | 1.34 |
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 160.49 us/op | 115.63 us/op | 1.39 |
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 280.16 us/op | 211.91 us/op | 1.32 |
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 438.40 us/op | 410.74 us/op | 1.07 |
prioritizePeers score 0:0 att 64-1 sync 4-1 | 537.09 us/op | 407.31 us/op | 1.32 |
RateTracker 1000000 limit, 1 obj count per request | 228.97 ns/op | 205.97 ns/op | 1.11 |
RateTracker 1000000 limit, 2 obj count per request | 172.25 ns/op | 151.92 ns/op | 1.13 |
RateTracker 1000000 limit, 4 obj count per request | 146.51 ns/op | 122.73 ns/op | 1.19 |
RateTracker 1000000 limit, 8 obj count per request | 132.07 ns/op | 105.59 ns/op | 1.25 |
RateTracker with prune | 5.2500 us/op | 4.3930 us/op | 1.20 |
array of 16000 items push then shift | 3.7764 us/op | 51.583 us/op | 0.07 |
LinkedList of 16000 items push then shift | 29.352 ns/op | 15.700 ns/op | 1.87 |
array of 16000 items push then pop | 279.17 ns/op | 236.92 ns/op | 1.18 |
LinkedList of 16000 items push then pop | 23.203 ns/op | 14.247 ns/op | 1.63 |
array of 24000 items push then shift | 5.4900 us/op | 77.340 us/op | 0.07 |
LinkedList of 24000 items push then shift | 31.317 ns/op | 19.377 ns/op | 1.62 |
array of 24000 items push then pop | 239.61 ns/op | 195.20 ns/op | 1.23 |
LinkedList of 24000 items push then pop | 23.940 ns/op | 15.871 ns/op | 1.51 |
intersect bitArray bitLen 8 | 13.495 ns/op | 10.662 ns/op | 1.27 |
intersect array and set length 8 | 198.12 ns/op | 147.58 ns/op | 1.34 |
intersect bitArray bitLen 128 | 71.797 ns/op | 57.891 ns/op | 1.24 |
intersect array and set length 128 | 2.7670 us/op | 1.9582 us/op | 1.41 |
pass gossip attestations to forkchoice per slot | 4.1547 ms/op | 3.1778 ms/op | 1.31 |
computeDeltas | 3.7545 ms/op | 3.2307 ms/op | 1.16 |
computeProposerBoostScoreFromBalances | 1.0679 ms/op | 809.15 us/op | 1.32 |
altair processAttestation - 250000 vs - 7PWei normalcase | 4.9216 ms/op | 3.7314 ms/op | 1.32 |
altair processAttestation - 250000 vs - 7PWei worstcase | 7.2761 ms/op | 5.4880 ms/op | 1.33 |
altair processAttestation - setStatus - 1/6 committees join | 272.08 us/op | 188.50 us/op | 1.44 |
altair processAttestation - setStatus - 1/3 committees join | 499.13 us/op | 366.12 us/op | 1.36 |
altair processAttestation - setStatus - 1/2 committees join | 746.74 us/op | 525.94 us/op | 1.42 |
altair processAttestation - setStatus - 2/3 committees join | 914.35 us/op | 694.14 us/op | 1.32 |
altair processAttestation - setStatus - 4/5 committees join | 1.3255 ms/op | 953.22 us/op | 1.39 |
altair processAttestation - setStatus - 100% committees join | 1.5045 ms/op | 1.1569 ms/op | 1.30 |
altair processBlock - 250000 vs - 7PWei normalcase | 31.715 ms/op | 27.696 ms/op | 1.15 |
altair processBlock - 250000 vs - 7PWei normalcase hashState | 45.694 ms/op | 33.378 ms/op | 1.37 |
altair processBlock - 250000 vs - 7PWei worstcase | 106.83 ms/op | 83.414 ms/op | 1.28 |
altair processBlock - 250000 vs - 7PWei worstcase hashState | 115.84 ms/op | 101.38 ms/op | 1.14 |
phase0 processBlock - 250000 vs - 7PWei normalcase | 5.6348 ms/op | 4.2065 ms/op | 1.34 |
phase0 processBlock - 250000 vs - 7PWei worstcase | 57.353 ms/op | 53.067 ms/op | 1.08 |
altair processEth1Data - 250000 vs - 7PWei normalcase | 1.0674 ms/op | 772.82 us/op | 1.38 |
Tree 40 250000 create | 904.50 ms/op | 762.78 ms/op | 1.19 |
Tree 40 250000 get(125000) | 345.69 ns/op | 251.43 ns/op | 1.37 |
Tree 40 250000 set(125000) | 2.8584 us/op | 2.6116 us/op | 1.09 |
Tree 40 250000 toArray() | 38.193 ms/op | 28.912 ms/op | 1.32 |
Tree 40 250000 iterate all - toArray() + loop | 37.231 ms/op | 29.976 ms/op | 1.24 |
Tree 40 250000 iterate all - get(i) | 129.41 ms/op | 110.78 ms/op | 1.17 |
MutableVector 250000 create | 19.383 ms/op | 12.439 ms/op | 1.56 |
MutableVector 250000 get(125000) | 15.405 ns/op | 10.770 ns/op | 1.43 |
MutableVector 250000 set(125000) | 744.97 ns/op | 472.55 ns/op | 1.58 |
MutableVector 250000 toArray() | 8.7337 ms/op | 5.8123 ms/op | 1.50 |
MutableVector 250000 iterate all - toArray() + loop | 8.8012 ms/op | 6.0507 ms/op | 1.45 |
MutableVector 250000 iterate all - get(i) | 4.1242 ms/op | 2.6775 ms/op | 1.54 |
Array 250000 create | 8.0988 ms/op | 5.5431 ms/op | 1.46 |
Array 250000 clone - spread | 4.9384 ms/op | 2.5564 ms/op | 1.93 |
Array 250000 get(125000) | 2.1530 ns/op | 1.2350 ns/op | 1.74 |
Array 250000 set(125000) | 2.0300 ns/op | 1.2300 ns/op | 1.65 |
Array 250000 iterate all - loop | 200.36 us/op | 151.27 us/op | 1.32 |
effectiveBalanceIncrements clone Uint8Array 300000 | 117.78 us/op | 76.720 us/op | 1.54 |
effectiveBalanceIncrements clone MutableVector 300000 | 1.5760 us/op | 611.00 ns/op | 2.58 |
effectiveBalanceIncrements rw all Uint8Array 300000 | 303.49 us/op | 247.55 us/op | 1.23 |
effectiveBalanceIncrements rw all MutableVector 300000 | 250.29 ms/op | 134.05 ms/op | 1.87 |
phase0 afterProcessEpoch - 250000 vs - 7PWei | 240.98 ms/op | 186.56 ms/op | 1.29 |
phase0 beforeProcessEpoch - 250000 vs - 7PWei | 108.32 ms/op | 58.308 ms/op | 1.86 |
altair processEpoch - mainnet_e81889 | 686.46 ms/op | 549.13 ms/op | 1.25 |
mainnet_e81889 - altair beforeProcessEpoch | 181.82 ms/op | 116.93 ms/op | 1.55 |
mainnet_e81889 - altair processJustificationAndFinalization | 38.120 us/op | 16.296 us/op | 2.34 |
mainnet_e81889 - altair processInactivityUpdates | 12.527 ms/op | 8.8829 ms/op | 1.41 |
mainnet_e81889 - altair processRewardsAndPenalties | 110.77 ms/op | 82.020 ms/op | 1.35 |
mainnet_e81889 - altair processRegistryUpdates | 9.3350 us/op | 2.6620 us/op | 3.51 |
mainnet_e81889 - altair processSlashings | 2.1940 us/op | 676.00 ns/op | 3.25 |
mainnet_e81889 - altair processEth1DataReset | 2.1490 us/op | 654.00 ns/op | 3.29 |
mainnet_e81889 - altair processEffectiveBalanceUpdates | 2.8660 ms/op | 1.9596 ms/op | 1.46 |
mainnet_e81889 - altair processSlashingsReset | 11.124 us/op | 4.2360 us/op | 2.63 |
mainnet_e81889 - altair processRandaoMixesReset | 12.856 us/op | 3.9440 us/op | 3.26 |
mainnet_e81889 - altair processHistoricalRootsUpdate | 2.2940 us/op | 588.00 ns/op | 3.90 |
mainnet_e81889 - altair processParticipationFlagUpdates | 6.7930 us/op | 3.0620 us/op | 2.22 |
mainnet_e81889 - altair processSyncCommitteeUpdates | 1.7740 us/op | 737.00 ns/op | 2.41 |
mainnet_e81889 - altair afterProcessEpoch | 230.64 ms/op | 218.18 ms/op | 1.06 |
phase0 processEpoch - mainnet_e58758 | 625.22 ms/op | 634.36 ms/op | 0.99 |
mainnet_e58758 - phase0 beforeProcessEpoch | 262.54 ms/op | 224.72 ms/op | 1.17 |
mainnet_e58758 - phase0 processJustificationAndFinalization | 34.194 us/op | 15.852 us/op | 2.16 |
mainnet_e58758 - phase0 processRewardsAndPenalties | 148.98 ms/op | 126.39 ms/op | 1.18 |
mainnet_e58758 - phase0 processRegistryUpdates | 19.322 us/op | 7.7070 us/op | 2.51 |
mainnet_e58758 - phase0 processSlashings | 2.0150 us/op | 620.00 ns/op | 3.25 |
mainnet_e58758 - phase0 processEth1DataReset | 2.3530 us/op | 611.00 ns/op | 3.85 |
mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 2.5884 ms/op | 1.8937 ms/op | 1.37 |
mainnet_e58758 - phase0 processSlashingsReset | 10.771 us/op | 4.5280 us/op | 2.38 |
mainnet_e58758 - phase0 processRandaoMixesReset | 13.452 us/op | 3.8470 us/op | 3.50 |
mainnet_e58758 - phase0 processHistoricalRootsUpdate | 2.2970 us/op | 647.00 ns/op | 3.55 |
mainnet_e58758 - phase0 processParticipationRecordUpdates | 12.246 us/op | 3.6570 us/op | 3.35 |
mainnet_e58758 - phase0 afterProcessEpoch | 187.85 ms/op | 162.71 ms/op | 1.15 |
phase0 processEffectiveBalanceUpdates - 250000 normalcase | 3.1145 ms/op | 2.1221 ms/op | 1.47 |
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 3.6001 ms/op | 2.2562 ms/op | 1.60 |
altair processInactivityUpdates - 250000 normalcase | 45.815 ms/op | 51.798 ms/op | 0.88 |
altair processInactivityUpdates - 250000 worstcase | 59.941 ms/op | 53.468 ms/op | 1.12 |
phase0 processRegistryUpdates - 250000 normalcase | 16.260 us/op | 6.7660 us/op | 2.40 |
phase0 processRegistryUpdates - 250000 badcase_full_deposits | 540.18 us/op | 374.51 us/op | 1.44 |
phase0 processRegistryUpdates - 250000 worstcase 0.5 | 250.07 ms/op | 227.53 ms/op | 1.10 |
altair processRewardsAndPenalties - 250000 normalcase | 155.16 ms/op | 134.16 ms/op | 1.16 |
altair processRewardsAndPenalties - 250000 worstcase | 101.92 ms/op | 135.14 ms/op | 0.75 |
phase0 getAttestationDeltas - 250000 normalcase | 16.377 ms/op | 11.835 ms/op | 1.38 |
phase0 getAttestationDeltas - 250000 worstcase | 13.938 ms/op | 11.712 ms/op | 1.19 |
phase0 processSlashings - 250000 worstcase | 6.2589 ms/op | 5.0034 ms/op | 1.25 |
altair processSyncCommitteeUpdates - 250000 | 339.18 ms/op | 289.03 ms/op | 1.17 |
BeaconState.hashTreeRoot - No change | 554.00 ns/op | 522.00 ns/op | 1.06 |
BeaconState.hashTreeRoot - 1 full validator | 70.272 us/op | 73.151 us/op | 0.96 |
BeaconState.hashTreeRoot - 32 full validator | 650.18 us/op | 718.76 us/op | 0.90 |
BeaconState.hashTreeRoot - 512 full validator | 7.4117 ms/op | 8.7571 ms/op | 0.85 |
BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 95.785 us/op | 101.28 us/op | 0.95 |
BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 1.6581 ms/op | 1.3132 ms/op | 1.26 |
BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 17.212 ms/op | 17.688 ms/op | 0.97 |
BeaconState.hashTreeRoot - 1 balances | 70.317 us/op | 67.073 us/op | 1.05 |
BeaconState.hashTreeRoot - 32 balances | 695.40 us/op | 735.50 us/op | 0.95 |
BeaconState.hashTreeRoot - 512 balances | 6.7723 ms/op | 7.0486 ms/op | 0.96 |
BeaconState.hashTreeRoot - 250000 balances | 112.05 ms/op | 118.12 ms/op | 0.95 |
aggregationBits - 2048 els - zipIndexesInBitList | 39.258 us/op | 25.255 us/op | 1.55 |
regular array get 100000 times | 79.561 us/op | 60.572 us/op | 1.31 |
wrappedArray get 100000 times | 81.017 us/op | 60.609 us/op | 1.34 |
arrayWithProxy get 100000 times | 33.891 ms/op | 28.945 ms/op | 1.17 |
ssz.Root.equals | 640.00 ns/op | 475.00 ns/op | 1.35 |
byteArrayEquals | 627.00 ns/op | 469.00 ns/op | 1.34 |
shuffle list - 16384 els | 13.117 ms/op | 11.293 ms/op | 1.16 |
shuffle list - 250000 els | 193.32 ms/op | 167.03 ms/op | 1.16 |
processSlot - 1 slots | 15.668 us/op | 13.003 us/op | 1.20 |
processSlot - 32 slots | 2.2081 ms/op | 1.9407 ms/op | 1.14 |
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 1.4798 ms/op | 382.59 us/op | 3.87 |
getCommitteeAssignments - req 1 vs - 250000 vc | 6.4451 ms/op | 5.4642 ms/op | 1.18 |
getCommitteeAssignments - req 100 vs - 250000 vc | 8.8400 ms/op | 7.9903 ms/op | 1.11 |
getCommitteeAssignments - req 1000 vs - 250000 vc | 9.4385 ms/op | 8.5769 ms/op | 1.10 |
computeProposers - vc 250000 | 19.717 ms/op | 18.440 ms/op | 1.07 |
computeEpochShuffling - vc 250000 | 197.24 ms/op | 170.07 ms/op | 1.16 |
getNextSyncCommittee - vc 250000 | 327.75 ms/op | 281.33 ms/op | 1.17 |
by benchmarkbot/action
nazarhussain
approved these changes
Jul 18, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Description
We are running way too many Lodestar nodes in e2e tests