Skip to content

Fix HttpClient event handler leak - #4107

Merged
wemeetagain merged 1 commit into
unstablefrom
dapplion/http-client-leak
Jun 3, 2022
Merged

Fix HttpClient event handler leak#4107
wemeetagain merged 1 commit into
unstablefrom
dapplion/http-client-leak

Conversation

@dapplion

@dapplion dapplion commented Jun 3, 2022

Copy link
Copy Markdown
Contributor

Motivation

Description

Remove the listener added and not a different function

Closes #4087

@dapplion
dapplion requested a review from a team as a code owner June 3, 2022 06:49
@dapplion dapplion mentioned this pull request Jun 3, 2022
Closed
@github-actions

github-actions Bot commented Jun 3, 2022

Copy link
Copy Markdown
Contributor

Performance Report

鉁旓笍 no performance regression detected

Full benchmark results
Benchmark suite Current: 9c852eb Previous: 86b95a7 Ratio
BeaconState.hashTreeRoot - No change 443.00 ns/op 515.00 ns/op 0.86
BeaconState.hashTreeRoot - 1 full validator 56.169 us/op 65.796 us/op 0.85
BeaconState.hashTreeRoot - 32 full validator 567.69 us/op 660.50 us/op 0.86
BeaconState.hashTreeRoot - 512 full validator 6.4936 ms/op 7.3084 ms/op 0.89
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 82.094 us/op 79.745 us/op 1.03
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.0966 ms/op 1.0972 ms/op 1.00
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 14.569 ms/op 15.323 ms/op 0.95
BeaconState.hashTreeRoot - 1 balances 57.805 us/op 62.076 us/op 0.93
BeaconState.hashTreeRoot - 32 balances 525.10 us/op 548.87 us/op 0.96
BeaconState.hashTreeRoot - 512 balances 4.7947 ms/op 5.3819 ms/op 0.89
BeaconState.hashTreeRoot - 250000 balances 108.09 ms/op 116.03 ms/op 0.93
processSlot - 1 slots 11.969 us/op 12.942 us/op 0.92
processSlot - 32 slots 1.7308 ms/op 1.9841 ms/op 0.87
getCommitteeAssignments - req 1 vs - 250000 vc 4.6472 ms/op 6.2772 ms/op 0.74
getCommitteeAssignments - req 100 vs - 250000 vc 6.4558 ms/op 8.6507 ms/op 0.75
getCommitteeAssignments - req 1000 vs - 250000 vc 7.0050 ms/op 9.1810 ms/op 0.76
computeProposers - vc 250000 15.838 ms/op 19.107 ms/op 0.83
computeEpochShuffling - vc 250000 151.37 ms/op 210.06 ms/op 0.72
getNextSyncCommittee - vc 250000 287.10 ms/op 309.98 ms/op 0.93
altair processAttestation - 250000 vs - 7PWei normalcase 4.3526 ms/op 4.7383 ms/op 0.92
altair processAttestation - 250000 vs - 7PWei worstcase 6.7872 ms/op 6.3026 ms/op 1.08
altair processAttestation - setStatus - 1/6 committees join 206.55 us/op 227.32 us/op 0.91
altair processAttestation - setStatus - 1/3 committees join 431.19 us/op 441.80 us/op 0.98
altair processAttestation - setStatus - 1/2 committees join 535.74 us/op 596.45 us/op 0.90
altair processAttestation - setStatus - 2/3 committees join 762.55 us/op 828.06 us/op 0.92
altair processAttestation - setStatus - 4/5 committees join 1.1479 ms/op 1.0903 ms/op 1.05
altair processAttestation - setStatus - 100% committees join 1.1087 ms/op 1.3160 ms/op 0.84
altair processBlock - 250000 vs - 7PWei normalcase 28.885 ms/op 30.952 ms/op 0.93
altair processBlock - 250000 vs - 7PWei normalcase hashState 40.780 ms/op 40.485 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase 86.943 ms/op 85.306 ms/op 1.02
altair processBlock - 250000 vs - 7PWei worstcase hashState 105.85 ms/op 126.02 ms/op 0.84
altair processEth1Data - 250000 vs - 7PWei normalcase 892.48 us/op 909.56 us/op 0.98
altair processEpoch - mainnet_e81889 592.10 ms/op 651.49 ms/op 0.91
mainnet_e81889 - altair beforeProcessEpoch 143.80 ms/op 157.90 ms/op 0.91
mainnet_e81889 - altair processJustificationAndFinalization 64.138 us/op 33.658 us/op 1.91
mainnet_e81889 - altair processInactivityUpdates 10.994 ms/op 13.104 ms/op 0.84
mainnet_e81889 - altair processRewardsAndPenalties 88.948 ms/op 104.28 ms/op 0.85
mainnet_e81889 - altair processRegistryUpdates 13.424 us/op 7.5380 us/op 1.78
mainnet_e81889 - altair processSlashings 4.9480 us/op 2.2950 us/op 2.16
mainnet_e81889 - altair processEth1DataReset 4.4480 us/op 2.3240 us/op 1.91
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.7015 ms/op 2.7126 ms/op 1.00
mainnet_e81889 - altair processSlashingsReset 21.334 us/op 10.037 us/op 2.13
mainnet_e81889 - altair processRandaoMixesReset 23.367 us/op 13.174 us/op 1.77
mainnet_e81889 - altair processHistoricalRootsUpdate 3.7180 us/op 2.1010 us/op 1.77
mainnet_e81889 - altair processParticipationFlagUpdates 11.787 us/op 6.1290 us/op 1.92
mainnet_e81889 - altair processSyncCommitteeUpdates 3.6060 us/op 1.8440 us/op 1.96
mainnet_e81889 - altair afterProcessEpoch 180.38 ms/op 226.60 ms/op 0.80
altair processInactivityUpdates - 250000 normalcase 49.292 ms/op 38.096 ms/op 1.29
altair processInactivityUpdates - 250000 worstcase 32.833 ms/op 31.543 ms/op 1.04
altair processRewardsAndPenalties - 250000 normalcase 82.850 ms/op 99.393 ms/op 0.83
altair processRewardsAndPenalties - 250000 worstcase 114.07 ms/op 93.426 ms/op 1.22
altair processSyncCommitteeUpdates - 250000 281.12 ms/op 333.86 ms/op 0.84
Tree 40 250000 create 905.25 ms/op 975.02 ms/op 0.93
Tree 40 250000 get(125000) 296.79 ns/op 340.98 ns/op 0.87
Tree 40 250000 set(125000) 2.6066 us/op 3.1945 us/op 0.82
Tree 40 250000 toArray() 32.946 ms/op 37.672 ms/op 0.87
Tree 40 250000 iterate all - toArray() + loop 35.418 ms/op 37.941 ms/op 0.93
Tree 40 250000 iterate all - get(i) 120.17 ms/op 133.60 ms/op 0.90
MutableVector 250000 create 15.397 ms/op 23.223 ms/op 0.66
MutableVector 250000 get(125000) 12.658 ns/op 16.242 ns/op 0.78
MutableVector 250000 set(125000) 643.49 ns/op 877.98 ns/op 0.73
MutableVector 250000 toArray() 6.1445 ms/op 8.5196 ms/op 0.72
MutableVector 250000 iterate all - toArray() + loop 5.9402 ms/op 8.7385 ms/op 0.68
MutableVector 250000 iterate all - get(i) 2.8588 ms/op 3.9928 ms/op 0.72
Array 250000 create 5.5039 ms/op 7.6094 ms/op 0.72
Array 250000 clone - spread 3.2140 ms/op 4.0493 ms/op 0.79
Array 250000 get(125000) 1.5130 ns/op 1.8290 ns/op 0.83
Array 250000 set(125000) 1.5310 ns/op 1.7980 ns/op 0.85
Array 250000 iterate all - loop 138.83 us/op 201.35 us/op 0.69
effectiveBalanceIncrements clone Uint8Array 300000 245.81 us/op 114.41 us/op 2.15
effectiveBalanceIncrements clone MutableVector 300000 773.00 ns/op 869.00 ns/op 0.89
effectiveBalanceIncrements rw all Uint8Array 300000 264.37 us/op 303.03 us/op 0.87
effectiveBalanceIncrements rw all MutableVector 300000 172.66 ms/op 210.38 ms/op 0.82
aggregationBits - 2048 els - zipIndexesInBitList 27.415 us/op 32.846 us/op 0.83
regular array get 100000 times 50.442 us/op 80.877 us/op 0.62
wrappedArray get 100000 times 48.423 us/op 81.040 us/op 0.60
arrayWithProxy get 100000 times 35.352 ms/op 45.026 ms/op 0.79
ssz.Root.equals 659.00 ns/op 607.00 ns/op 1.09
byteArrayEquals 514.00 ns/op 618.00 ns/op 0.83
phase0 processBlock - 250000 vs - 7PWei normalcase 4.1681 ms/op 4.1795 ms/op 1.00
phase0 processBlock - 250000 vs - 7PWei worstcase 44.170 ms/op 53.770 ms/op 0.82
phase0 afterProcessEpoch - 250000 vs - 7PWei 168.11 ms/op 220.68 ms/op 0.76
phase0 beforeProcessEpoch - 250000 vs - 7PWei 74.432 ms/op 79.551 ms/op 0.94
phase0 processEpoch - mainnet_e58758 517.10 ms/op 666.93 ms/op 0.78
mainnet_e58758 - phase0 beforeProcessEpoch 245.34 ms/op 258.91 ms/op 0.95
mainnet_e58758 - phase0 processJustificationAndFinalization 69.098 us/op 34.877 us/op 1.98
mainnet_e58758 - phase0 processRewardsAndPenalties 122.48 ms/op 83.616 ms/op 1.46
mainnet_e58758 - phase0 processRegistryUpdates 32.311 us/op 18.388 us/op 1.76
mainnet_e58758 - phase0 processSlashings 3.6510 us/op 2.7160 us/op 1.34
mainnet_e58758 - phase0 processEth1DataReset 3.9510 us/op 2.0120 us/op 1.96
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.0802 ms/op 2.4321 ms/op 0.86
mainnet_e58758 - phase0 processSlashingsReset 20.512 us/op 9.3210 us/op 2.20
mainnet_e58758 - phase0 processRandaoMixesReset 24.454 us/op 12.438 us/op 1.97
mainnet_e58758 - phase0 processHistoricalRootsUpdate 5.3980 us/op 2.5550 us/op 2.11
mainnet_e58758 - phase0 processParticipationRecordUpdates 26.414 us/op 13.797 us/op 1.91
mainnet_e58758 - phase0 afterProcessEpoch 143.58 ms/op 195.10 ms/op 0.74
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.0193 ms/op 3.5495 ms/op 0.57
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.2540 ms/op 3.6973 ms/op 0.61
phase0 processRegistryUpdates - 250000 normalcase 28.561 us/op 13.942 us/op 2.05
phase0 processRegistryUpdates - 250000 badcase_full_deposits 438.82 us/op 505.10 us/op 0.87
phase0 processRegistryUpdates - 250000 worstcase 0.5 245.17 ms/op 278.81 ms/op 0.88
phase0 getAttestationDeltas - 250000 normalcase 14.640 ms/op 17.523 ms/op 0.84
phase0 getAttestationDeltas - 250000 worstcase 11.902 ms/op 17.899 ms/op 0.66
phase0 processSlashings - 250000 worstcase 5.8223 ms/op 6.4321 ms/op 0.91
shuffle list - 16384 els 11.449 ms/op 13.133 ms/op 0.87
shuffle list - 250000 els 150.79 ms/op 192.99 ms/op 0.78
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 452.28 us/op 433.91 us/op 1.04
pass gossip attestations to forkchoice per slot 4.1580 ms/op 3.8408 ms/op 1.08
computeDeltas 3.8360 ms/op 3.6020 ms/op 1.06
computeProposerBoostScoreFromBalances 751.42 us/op 1.1210 ms/op 0.67
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.5337 ms/op 2.5824 ms/op 0.98
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 77.030 us/op 111.94 us/op 0.69
BLS verify - blst-native 1.9476 ms/op 2.2240 ms/op 0.88
BLS verifyMultipleSignatures 3 - blst-native 3.8414 ms/op 4.5386 ms/op 0.85
BLS verifyMultipleSignatures 8 - blst-native 8.0555 ms/op 9.7893 ms/op 0.82
BLS verifyMultipleSignatures 32 - blst-native 30.848 ms/op 35.034 ms/op 0.88
BLS aggregatePubkeys 32 - blst-native 42.098 us/op 46.239 us/op 0.91
BLS aggregatePubkeys 128 - blst-native 180.58 us/op 179.38 us/op 1.01
getAttestationsForBlock 73.705 ms/op 82.324 ms/op 0.90
isKnown best case - 1 super set check 524.00 ns/op 524.00 ns/op 1.00
isKnown normal case - 2 super set checks 500.00 ns/op 525.00 ns/op 0.95
isKnown worse case - 16 super set checks 529.00 ns/op 526.00 ns/op 1.01
CheckpointStateCache - add get delete 12.942 us/op 12.250 us/op 1.06
validate gossip signedAggregateAndProof - struct 4.3663 ms/op 4.9065 ms/op 0.89
validate gossip attestation - struct 2.0809 ms/op 2.3180 ms/op 0.90
altair verifyImport mainnet_s3766816:31 6.5462 s/op 7.3280 s/op 0.89
pickEth1Vote - no votes 2.1015 ms/op 2.5187 ms/op 0.83
pickEth1Vote - max votes 26.596 ms/op 29.992 ms/op 0.89
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.241 ms/op 14.832 ms/op 0.89
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.656 ms/op 28.578 ms/op 0.76
pickEth1Vote - Eth1Data fastSerialize value x2048 1.5539 ms/op 1.8334 ms/op 0.85
pickEth1Vote - Eth1Data fastSerialize tree x2048 19.585 ms/op 21.276 ms/op 0.92
bytes32 toHexString 1.0430 us/op 1.3660 us/op 0.76
bytes32 Buffer.toString(hex) 686.00 ns/op 851.00 ns/op 0.81
bytes32 Buffer.toString(hex) from Uint8Array 934.00 ns/op 1.1050 us/op 0.85
bytes32 Buffer.toString(hex) + 0x 685.00 ns/op 856.00 ns/op 0.80
Object access 1 prop 0.40200 ns/op 0.47500 ns/op 0.85
Map access 1 prop 0.30900 ns/op 0.36100 ns/op 0.86
Object get x1000 15.032 ns/op 21.542 ns/op 0.70
Map get x1000 0.91600 ns/op 1.2450 ns/op 0.74
Object set x1000 96.135 ns/op 141.30 ns/op 0.68
Map set x1000 71.774 ns/op 88.881 ns/op 0.81
Return object 10000 times 0.39630 ns/op 0.45030 ns/op 0.88
Throw Error 10000 times 5.8575 us/op 6.9518 us/op 0.84
enrSubnets - fastDeserialize 64 bits 2.6320 us/op 3.3340 us/op 0.79
enrSubnets - ssz BitVector 64 bits 788.00 ns/op 957.00 ns/op 0.82
enrSubnets - fastDeserialize 4 bits 391.00 ns/op 505.00 ns/op 0.77
enrSubnets - ssz BitVector 4 bits 715.00 ns/op 921.00 ns/op 0.78
prioritizePeers score -10:0 att 32-0.1 sync 2-0 105.19 us/op 122.95 us/op 0.86
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 149.26 us/op 165.28 us/op 0.90
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 261.09 us/op 278.93 us/op 0.94
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 474.45 us/op 455.75 us/op 1.04
prioritizePeers score 0:0 att 64-1 sync 4-1 499.53 us/op 547.81 us/op 0.91
RateTracker 1000000 limit, 1 obj count per request 190.40 ns/op 230.06 ns/op 0.83
RateTracker 1000000 limit, 2 obj count per request 140.14 ns/op 175.22 ns/op 0.80
RateTracker 1000000 limit, 4 obj count per request 118.24 ns/op 146.08 ns/op 0.81
RateTracker 1000000 limit, 8 obj count per request 106.74 ns/op 133.36 ns/op 0.80
RateTracker with prune 5.1550 us/op 5.7440 us/op 0.90
array of 16000 items push then shift 5.0133 us/op 3.7519 us/op 1.34
LinkedList of 16000 items push then shift 28.150 ns/op 28.946 ns/op 0.97
array of 16000 items push then pop 261.74 ns/op 266.81 ns/op 0.98
LinkedList of 16000 items push then pop 22.983 ns/op 23.583 ns/op 0.97
array of 24000 items push then shift 7.9473 us/op 5.4226 us/op 1.47
LinkedList of 24000 items push then shift 26.969 ns/op 29.516 ns/op 0.91
array of 24000 items push then pop 226.19 ns/op 251.06 ns/op 0.90
LinkedList of 24000 items push then pop 25.220 ns/op 24.650 ns/op 1.02
intersect bitArray bitLen 8 13.148 ns/op 13.915 ns/op 0.94
intersect array and set length 8 214.22 ns/op 213.87 ns/op 1.00
intersect bitArray bitLen 128 71.866 ns/op 86.012 ns/op 0.84
intersect array and set length 128 2.6282 us/op 2.8186 us/op 0.93

by benchmarkbot/action

@wemeetagain
wemeetagain merged commit a61e8a5 into unstable Jun 3, 2022
@wemeetagain
wemeetagain deleted the dapplion/http-client-leak branch June 3, 2022 13:24
g11tech pushed a commit that referenced this pull request Jun 3, 2022
@dapplion @g11tech
Fix HttpClient event handler leak (#4107)
9bb9127 
Fix leak in signalGlobal event listeners
@dapplion

dapplion commented Jun 4, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Ropsten deployment also confirms that this PR fixes the resource leak 馃憤 Ropsten nodes are running 2000-4000 keys each so this issue is the most visible there due to the high number of vc - bn interactions

Screenshot from 2022-06-04 18-24-23

wemeetagain added a commit that referenced this pull request Jun 7, 2022
@wemeetagain @g11tech @dapplion
Stable (#4129)
56a6bb4 
* v0.38.0

* Populate eth1 deposit cache with remote follow distance timed based (#4111)

* Fix HttpClient event handler leak (#4107)

Fix leak in signalGlobal event listeners

* Add Eth1DepositTracker metrics (#4110)

* Add Eth1DepositTracker metrics

* Add Eth1DepositTracker metrics

* Remove duplicate line

* Fix publish stable workflow (#4130)

Co-authored-by: harkamal <gajinder@g11.in>
Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
@kalbroni7 kalbroni7 mentioned this pull request Jul 14, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes
@twoeths twoeths twoeths approved these changes
+1 more reviewer
@g11tech g11tech g11tech approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Resource leak in validator client

4 participants

@dapplion @wemeetagain @twoeths @g11tech