Skip to content

Fix PeerScore.update() function - #5094

Merged
wemeetagain merged 4 commits into
unstablefrom
tuyen/update_score
Feb 6, 2023
Merged

Fix PeerScore.update() function#5094
wemeetagain merged 4 commits into
unstablefrom
tuyen/update_score

Conversation

@twoeths

@twoeths twoeths commented Feb 2, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Motivation

  1. Some incorrect computation where final score is 0, lodestar score is 0 and gossipScore < 0 as below:
{"peer_id":"16Uiu2HAmVcU5cjAGqRwCM1PyniSbhMPEEAgCM5aJ6qZwSFfsDgX5","lodestar_score":0,"gossip_score":-6810.915930589742,"ignore_negative_gossip_score":false,"score":0,"last_update":1675327150937}

  1. Right now we ignore 5 biggest negative gossip score to allow them to recover but some of them are very small, for example -6000

  2. We have no idea how clients perform in our peers

Description

  1. In the above case the final score should be < 0, should remove the small optimization to check lodestarScore !== 0 (which was done before we take gossipsub score into account)
  2. Only ignore negative gossipsub if it's > -1000
  3. Track scores by client

part of #5074

@twoeths
twoeths requested a review from a team as a code owner February 2, 2023 10:41
@github-actions

github-actions Bot commented Feb 2, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Performance Report

鉁旓笍 no performance regression detected

Full benchmark results
Benchmark suite Current: b67d3e6 Previous: 6793305 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 606.65 us/op 518.29 us/op 1.17
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 62.174 us/op 46.398 us/op 1.34
BLS verify - blst-native 1.3658 ms/op 1.2119 ms/op 1.13
BLS verifyMultipleSignatures 3 - blst-native 2.7849 ms/op 2.4720 ms/op 1.13
BLS verifyMultipleSignatures 8 - blst-native 6.0301 ms/op 5.3799 ms/op 1.12
BLS verifyMultipleSignatures 32 - blst-native 22.193 ms/op 19.388 ms/op 1.14
BLS aggregatePubkeys 32 - blst-native 29.957 us/op 26.034 us/op 1.15
BLS aggregatePubkeys 128 - blst-native 115.84 us/op 101.77 us/op 1.14
getAttestationsForBlock 69.716 ms/op 55.973 ms/op 1.25
isKnown best case - 1 super set check 326.00 ns/op 274.00 ns/op 1.19
isKnown normal case - 2 super set checks 319.00 ns/op 264.00 ns/op 1.21
isKnown worse case - 16 super set checks 321.00 ns/op 263.00 ns/op 1.22
CheckpointStateCache - add get delete 6.6350 us/op 6.0110 us/op 1.10
validate gossip signedAggregateAndProof - struct 3.1114 ms/op 2.7958 ms/op 1.11
validate gossip attestation - struct 1.4559 ms/op 1.3385 ms/op 1.09
pickEth1Vote - no votes 1.4967 ms/op 1.3523 ms/op 1.11
pickEth1Vote - max votes 12.571 ms/op 9.7881 ms/op 1.28
pickEth1Vote - Eth1Data hashTreeRoot value x2048 10.233 ms/op 8.8528 ms/op 1.16
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 17.534 ms/op 14.371 ms/op 1.22
pickEth1Vote - Eth1Data fastSerialize value x2048 798.40 us/op 662.21 us/op 1.21
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.8767 ms/op 4.7295 ms/op 1.67
bytes32 toHexString 764.00 ns/op 503.00 ns/op 1.52
bytes32 Buffer.toString(hex) 455.00 ns/op 354.00 ns/op 1.29
bytes32 Buffer.toString(hex) from Uint8Array 659.00 ns/op 576.00 ns/op 1.14
bytes32 Buffer.toString(hex) + 0x 486.00 ns/op 348.00 ns/op 1.40
Object access 1 prop 0.24900 ns/op 0.17600 ns/op 1.41
Map access 1 prop 0.21400 ns/op 0.16500 ns/op 1.30
Object get x1000 7.3470 ns/op 6.9600 ns/op 1.06
Map get x1000 0.71300 ns/op 0.64800 ns/op 1.10
Object set x1000 69.002 ns/op 55.808 ns/op 1.24
Map set x1000 56.375 ns/op 45.509 ns/op 1.24
Return object 10000 times 0.26100 ns/op 0.24060 ns/op 1.08
Throw Error 10000 times 4.5455 us/op 4.1654 us/op 1.09
fastMsgIdFn sha256 / 200 bytes 3.8770 us/op 3.5030 us/op 1.11
fastMsgIdFn h32 xxhash / 200 bytes 382.00 ns/op 294.00 ns/op 1.30
fastMsgIdFn h64 xxhash / 200 bytes 548.00 ns/op 407.00 ns/op 1.35
fastMsgIdFn sha256 / 1000 bytes 12.648 us/op 11.786 us/op 1.07
fastMsgIdFn h32 xxhash / 1000 bytes 505.00 ns/op 435.00 ns/op 1.16
fastMsgIdFn h64 xxhash / 1000 bytes 593.00 ns/op 490.00 ns/op 1.21
fastMsgIdFn sha256 / 10000 bytes 112.62 us/op 104.91 us/op 1.07
fastMsgIdFn h32 xxhash / 10000 bytes 2.1150 us/op 1.9850 us/op 1.07
fastMsgIdFn h64 xxhash / 10000 bytes 1.4520 us/op 1.3830 us/op 1.05
enrSubnets - fastDeserialize 64 bits 1.6040 us/op 1.2830 us/op 1.25
enrSubnets - ssz BitVector 64 bits 637.00 ns/op 490.00 ns/op 1.30
enrSubnets - fastDeserialize 4 bits 226.00 ns/op 176.00 ns/op 1.28
enrSubnets - ssz BitVector 4 bits 645.00 ns/op 496.00 ns/op 1.30
prioritizePeers score -10:0 att 32-0.1 sync 2-0 106.61 us/op 97.228 us/op 1.10
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 143.78 us/op 134.17 us/op 1.07
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 216.06 us/op 190.41 us/op 1.13
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 354.37 us/op 362.76 us/op 0.98
prioritizePeers score 0:0 att 64-1 sync 4-1 397.87 us/op 414.90 us/op 0.96
array of 16000 items push then shift 1.7147 us/op 1.6785 us/op 1.02
LinkedList of 16000 items push then shift 9.3700 ns/op 9.1830 ns/op 1.02
array of 16000 items push then pop 98.318 ns/op 115.78 ns/op 0.85
LinkedList of 16000 items push then pop 9.0610 ns/op 9.0360 ns/op 1.00
array of 24000 items push then shift 2.4270 us/op 2.3943 us/op 1.01
LinkedList of 24000 items push then shift 9.2760 ns/op 9.2490 ns/op 1.00
array of 24000 items push then pop 85.561 ns/op 85.502 ns/op 1.00
LinkedList of 24000 items push then pop 8.8240 ns/op 9.0480 ns/op 0.98
intersect bitArray bitLen 8 15.344 ns/op 13.187 ns/op 1.16
intersect array and set length 8 84.193 ns/op 89.776 ns/op 0.94
intersect bitArray bitLen 128 45.272 ns/op 44.007 ns/op 1.03
intersect array and set length 128 1.1148 us/op 1.2529 us/op 0.89
Buffer.concat 32 items 2.8850 us/op 2.7550 us/op 1.05
Uint8Array.set 32 items 3.1260 us/op 2.3090 us/op 1.35
pass gossip attestations to forkchoice per slot 2.4058 ms/op 4.1862 ms/op 0.57
computeDeltas 2.9131 ms/op 3.0512 ms/op 0.95
computeProposerBoostScoreFromBalances 1.8019 ms/op 1.8142 ms/op 0.99
altair processAttestation - 250000 vs - 7PWei normalcase 2.1491 ms/op 2.5599 ms/op 0.84
altair processAttestation - 250000 vs - 7PWei worstcase 3.5060 ms/op 3.9573 ms/op 0.89
altair processAttestation - setStatus - 1/6 committees join 143.04 us/op 140.25 us/op 1.02
altair processAttestation - setStatus - 1/3 committees join 280.52 us/op 279.63 us/op 1.00
altair processAttestation - setStatus - 1/2 committees join 384.71 us/op 372.66 us/op 1.03
altair processAttestation - setStatus - 2/3 committees join 472.41 us/op 464.64 us/op 1.02
altair processAttestation - setStatus - 4/5 committees join 648.95 us/op 646.82 us/op 1.00
altair processAttestation - setStatus - 100% committees join 768.97 us/op 759.52 us/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase 15.886 ms/op 16.388 ms/op 0.97
altair processBlock - 250000 vs - 7PWei normalcase hashState 24.043 ms/op 24.709 ms/op 0.97
altair processBlock - 250000 vs - 7PWei worstcase 51.636 ms/op 49.659 ms/op 1.04
altair processBlock - 250000 vs - 7PWei worstcase hashState 73.734 ms/op 63.062 ms/op 1.17
phase0 processBlock - 250000 vs - 7PWei normalcase 2.1198 ms/op 1.9034 ms/op 1.11
phase0 processBlock - 250000 vs - 7PWei worstcase 29.352 ms/op 27.749 ms/op 1.06
altair processEth1Data - 250000 vs - 7PWei normalcase 502.60 us/op 446.19 us/op 1.13
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 8.5210 us/op 6.9670 us/op 1.22
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 26.364 us/op 20.343 us/op 1.30
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 8.7780 us/op 8.2660 us/op 1.06
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 7.5930 us/op 7.1890 us/op 1.06
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 90.304 us/op 80.662 us/op 1.12
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 697.90 us/op 636.55 us/op 1.10
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 916.18 us/op 894.89 us/op 1.02
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 914.23 us/op 854.96 us/op 1.07
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.4283 ms/op 2.2896 ms/op 1.06
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.5916 ms/op 1.5219 ms/op 1.05
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 4.3197 ms/op 3.8296 ms/op 1.13
Tree 40 250000 create 360.96 ms/op 289.24 ms/op 1.25
Tree 40 250000 get(125000) 195.25 ns/op 177.83 ns/op 1.10
Tree 40 250000 set(125000) 1.0520 us/op 889.25 ns/op 1.18
Tree 40 250000 toArray() 22.290 ms/op 17.563 ms/op 1.27
Tree 40 250000 iterate all - toArray() + loop 23.005 ms/op 17.659 ms/op 1.30
Tree 40 250000 iterate all - get(i) 75.657 ms/op 65.891 ms/op 1.15
MutableVector 250000 create 9.7743 ms/op 10.877 ms/op 0.90
MutableVector 250000 get(125000) 6.6450 ns/op 6.2750 ns/op 1.06
MutableVector 250000 set(125000) 267.85 ns/op 279.07 ns/op 0.96
MutableVector 250000 toArray() 3.1940 ms/op 3.0755 ms/op 1.04
MutableVector 250000 iterate all - toArray() + loop 3.4939 ms/op 3.4278 ms/op 1.02
MutableVector 250000 iterate all - get(i) 1.5634 ms/op 1.4983 ms/op 1.04
Array 250000 create 2.8934 ms/op 2.7139 ms/op 1.07
Array 250000 clone - spread 1.2512 ms/op 1.0748 ms/op 1.16
Array 250000 get(125000) 0.64500 ns/op 0.52800 ns/op 1.22
Array 250000 set(125000) 0.70900 ns/op 0.59800 ns/op 1.19
Array 250000 iterate all - loop 108.40 us/op 109.33 us/op 0.99
effectiveBalanceIncrements clone Uint8Array 300000 35.704 us/op 27.912 us/op 1.28
effectiveBalanceIncrements clone MutableVector 300000 421.00 ns/op 314.00 ns/op 1.34
effectiveBalanceIncrements rw all Uint8Array 300000 175.79 us/op 164.46 us/op 1.07
effectiveBalanceIncrements rw all MutableVector 300000 85.905 ms/op 76.221 ms/op 1.13
phase0 afterProcessEpoch - 250000 vs - 7PWei 118.61 ms/op 110.98 ms/op 1.07
phase0 beforeProcessEpoch - 250000 vs - 7PWei 36.204 ms/op 43.363 ms/op 0.83
altair processEpoch - mainnet_e81889 337.19 ms/op 310.90 ms/op 1.08
mainnet_e81889 - altair beforeProcessEpoch 63.354 ms/op 52.134 ms/op 1.22
mainnet_e81889 - altair processJustificationAndFinalization 17.673 us/op 19.173 us/op 0.92
mainnet_e81889 - altair processInactivityUpdates 5.4874 ms/op 5.8954 ms/op 0.93
mainnet_e81889 - altair processRewardsAndPenalties 49.749 ms/op 68.933 ms/op 0.72
mainnet_e81889 - altair processRegistryUpdates 2.0880 us/op 2.5200 us/op 0.83
mainnet_e81889 - altair processSlashings 425.00 ns/op 871.00 ns/op 0.49
mainnet_e81889 - altair processEth1DataReset 438.00 ns/op 566.00 ns/op 0.77
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2569 ms/op 1.2343 ms/op 1.02
mainnet_e81889 - altair processSlashingsReset 4.2850 us/op 4.9820 us/op 0.86
mainnet_e81889 - altair processRandaoMixesReset 4.0880 us/op 6.4100 us/op 0.64
mainnet_e81889 - altair processHistoricalRootsUpdate 578.00 ns/op 1.0560 us/op 0.55
mainnet_e81889 - altair processParticipationFlagUpdates 2.1700 us/op 3.0650 us/op 0.71
mainnet_e81889 - altair processSyncCommitteeUpdates 403.00 ns/op 831.00 ns/op 0.48
mainnet_e81889 - altair afterProcessEpoch 117.91 ms/op 129.64 ms/op 0.91
phase0 processEpoch - mainnet_e58758 320.94 ms/op 453.30 ms/op 0.71
mainnet_e58758 - phase0 beforeProcessEpoch 126.42 ms/op 139.36 ms/op 0.91
mainnet_e58758 - phase0 processJustificationAndFinalization 16.522 us/op 17.670 us/op 0.94
mainnet_e58758 - phase0 processRewardsAndPenalties 56.924 ms/op 65.780 ms/op 0.87
mainnet_e58758 - phase0 processRegistryUpdates 7.3980 us/op 9.9490 us/op 0.74
mainnet_e58758 - phase0 processSlashings 611.00 ns/op 550.00 ns/op 1.11
mainnet_e58758 - phase0 processEth1DataReset 539.00 ns/op 616.00 ns/op 0.88
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0093 ms/op 1.0018 ms/op 1.01
mainnet_e58758 - phase0 processSlashingsReset 2.9360 us/op 3.6040 us/op 0.81
mainnet_e58758 - phase0 processRandaoMixesReset 4.4330 us/op 4.3220 us/op 1.03
mainnet_e58758 - phase0 processHistoricalRootsUpdate 954.00 ns/op 546.00 ns/op 1.75
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.1550 us/op 4.0130 us/op 1.04
mainnet_e58758 - phase0 afterProcessEpoch 92.852 ms/op 99.063 ms/op 0.94
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2503 ms/op 1.2388 ms/op 1.01
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4785 ms/op 1.5176 ms/op 0.97
altair processInactivityUpdates - 250000 normalcase 24.757 ms/op 19.940 ms/op 1.24
altair processInactivityUpdates - 250000 worstcase 26.073 ms/op 27.064 ms/op 0.96
phase0 processRegistryUpdates - 250000 normalcase 7.0180 us/op 7.1780 us/op 0.98
phase0 processRegistryUpdates - 250000 badcase_full_deposits 245.03 us/op 269.31 us/op 0.91
phase0 processRegistryUpdates - 250000 worstcase 0.5 121.38 ms/op 114.01 ms/op 1.06
altair processRewardsAndPenalties - 250000 normalcase 66.683 ms/op 66.560 ms/op 1.00
altair processRewardsAndPenalties - 250000 worstcase 68.076 ms/op 85.659 ms/op 0.79
phase0 getAttestationDeltas - 250000 normalcase 6.6564 ms/op 13.268 ms/op 0.50
phase0 getAttestationDeltas - 250000 worstcase 6.6455 ms/op 11.472 ms/op 0.58
phase0 processSlashings - 250000 worstcase 3.6687 ms/op 5.2295 ms/op 0.70
altair processSyncCommitteeUpdates - 250000 182.17 ms/op 267.11 ms/op 0.68
BeaconState.hashTreeRoot - No change 288.00 ns/op 272.00 ns/op 1.06
BeaconState.hashTreeRoot - 1 full validator 56.766 us/op 56.044 us/op 1.01
BeaconState.hashTreeRoot - 32 full validator 579.41 us/op 555.32 us/op 1.04
BeaconState.hashTreeRoot - 512 full validator 5.3183 ms/op 5.1107 ms/op 1.04
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 64.353 us/op 64.968 us/op 0.99
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 903.78 us/op 900.49 us/op 1.00
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 12.313 ms/op 12.402 ms/op 0.99
BeaconState.hashTreeRoot - 1 balances 49.936 us/op 49.128 us/op 1.02
BeaconState.hashTreeRoot - 32 balances 478.11 us/op 483.02 us/op 0.99
BeaconState.hashTreeRoot - 512 balances 4.5095 ms/op 4.8514 ms/op 0.93
BeaconState.hashTreeRoot - 250000 balances 75.113 ms/op 77.197 ms/op 0.97
aggregationBits - 2048 els - zipIndexesInBitList 17.296 us/op 18.213 us/op 0.95
regular array get 100000 times 48.457 us/op 40.968 us/op 1.18
wrappedArray get 100000 times 34.036 us/op 33.735 us/op 1.01
arrayWithProxy get 100000 times 16.950 ms/op 15.639 ms/op 1.08
ssz.Root.equals 575.00 ns/op 611.00 ns/op 0.94
byteArrayEquals 568.00 ns/op 587.00 ns/op 0.97
shuffle list - 16384 els 7.0344 ms/op 6.8495 ms/op 1.03
shuffle list - 250000 els 103.30 ms/op 100.60 ms/op 1.03
processSlot - 1 slots 9.0220 us/op 9.1180 us/op 0.99
processSlot - 32 slots 1.3636 ms/op 1.3833 ms/op 0.99
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 203.74 us/op 204.65 us/op 1.00
getCommitteeAssignments - req 1 vs - 250000 vc 3.0082 ms/op 2.9448 ms/op 1.02
getCommitteeAssignments - req 100 vs - 250000 vc 4.3170 ms/op 4.1345 ms/op 1.04
getCommitteeAssignments - req 1000 vs - 250000 vc 4.6394 ms/op 4.4733 ms/op 1.04
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.1900 ns/op 4.9600 ns/op 1.05
state getBlockRootAtSlot - 250000 vs - 7PWei 977.21 ns/op 1.0615 us/op 0.92
computeProposers - vc 250000 11.739 ms/op 10.780 ms/op 1.09
computeEpochShuffling - vc 250000 107.51 ms/op 102.74 ms/op 1.05
getNextSyncCommittee - vc 250000 193.59 ms/op 171.52 ms/op 1.13

by benchmarkbot/action

@twoeths
twoeths marked this pull request as draft February 2, 2023 23:13
@twoeths
twoeths marked this pull request as ready for review February 6, 2023 01:28
@twoeths

twoeths commented Feb 6, 2023

Copy link
Copy Markdown
Member Author

It shows that Nimbus gossipsub score is low on mainnet

Screen Shot 2023-02-06 at 08 34 21

while other clients perform well, like Lighthouse

Screen Shot 2023-02-06 at 08 34 50

@twoeths twoeths mentioned this pull request Feb 6, 2023
Closed
@wemeetagain
wemeetagain merged commit bc77596 into unstable Feb 6, 2023
@wemeetagain
wemeetagain deleted the tuyen/update_score branch February 6, 2023 08:29
wemeetagain pushed a commit that referenced this pull request Feb 6, 2023
@twoeths @wemeetagain
Fix PeerScore.update() function (#5094)
68c82ba 
* Fix PeerScore.update() function

* Introduce negativeGossipScoreIgnoreThreshold

* Track peer scores by client

* Reset gossip score by client
@wemeetagain wemeetagain mentioned this pull request Feb 6, 2023
Merged
@wemeetagain

wemeetagain commented Mar 21, 2023

Copy link
Copy Markdown
Member

馃帀 This PR is included in v1.6.0 馃帀

@nflaig nflaig mentioned this pull request Apr 13, 2026
Merged
@ensi321 ensi321 mentioned this pull request Apr 14, 2026
Merged
4 tasks
@twoeths twoeths mentioned this pull request Apr 15, 2026
Closed
5 tasks
@gemini-code-assist gemini-code-assist Bot mentioned this pull request Apr 22, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@twoeths @wemeetagain