Skip to content

fix: gracefully terminate connections when closing http server - #5786

Merged
wemeetagain merged 7 commits into
unstablefrom
nflaig/server-graceful-shutdown
Jul 24, 2023
Merged

fix: gracefully terminate connections when closing http server#5786
wemeetagain merged 7 commits into
unstablefrom
nflaig/server-graceful-shutdown

Conversation

@nflaig

@nflaig nflaig commented Jul 21, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Motivation

While looking into #5783 I noticed that our REST API server is not really "friendly" to clients, it abruptly terminates connections.

We should give the beacon node some time to complete pending requests and only forcefully close connections if timeout is reached. This improves client experience and prevents potential data corruption if request is writing data.

Description

Gracefully close HTTP server by waiting for all connections to drain until timeout

  • it immediately destroys all sockets without an attached HTTP request
  • it allows graceful timeout to sockets with ongoing HTTP requests
  • it informs connections using keep-alive that server is shutting down by setting a Connection: close header
  • it forcefully terminates all connections after timeout (1 second) is reached

Further considerations

It might make sense to close server immediately to stop accepting new connections.

Right now, what happens is that while terminating, new connections are just force closed by destroying the socket.

lodestar/packages/beacon-node/src/api/rest/activeSockets.ts

Lines 28 to 29 in 485a996

if (this.terminated) {
socket.destroy(Error("Closing"));

This causes curl: (56) Recv failure: Connection reset by peer error on the client.

If the server is closed immediately it will cause curl: (7) Failed to connect to localhost port 9596 after 0 ms: Connection refused on the client as server stops listening for connections.

The difference is likely insignificant in our case as the shutdown period is really short and the beacon node generally does not receive that many requests per second and in both cases the client will receive an error.

This has been discussed in http-terminator repository already gajus/http-terminator#22, but since our current and now updated implementation follows that of http-terminator closely, I kept the implementation to close server after terminating sockets.

@nflaig
nflaig requested a review from a team as a code owner July 21, 2023 21:55
@github-actions

github-actions Bot commented Jul 21, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Performance Report

鉁旓笍 no performance regression detected

Full benchmark results
Benchmark suite Current: f4becee Previous: 3257345 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 776.95 us/op 847.56 us/op 0.92
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 74.300 us/op 92.416 us/op 0.80
BLS verify - blst-native 1.1788 ms/op 1.4285 ms/op 0.83
BLS verifyMultipleSignatures 3 - blst-native 2.4020 ms/op 2.8654 ms/op 0.84
BLS verifyMultipleSignatures 8 - blst-native 5.1521 ms/op 6.1024 ms/op 0.84
BLS verifyMultipleSignatures 32 - blst-native 18.605 ms/op 22.164 ms/op 0.84
BLS aggregatePubkeys 32 - blst-native 24.599 us/op 29.186 us/op 0.84
BLS aggregatePubkeys 128 - blst-native 96.818 us/op 114.69 us/op 0.84
getAttestationsForBlock 48.671 ms/op 64.646 ms/op 0.75
isKnown best case - 1 super set check 266.00 ns/op 347.00 ns/op 0.77
isKnown normal case - 2 super set checks 259.00 ns/op 342.00 ns/op 0.76
isKnown worse case - 16 super set checks 259.00 ns/op 331.00 ns/op 0.78
CheckpointStateCache - add get delete 4.5840 us/op 5.8660 us/op 0.78
validate api signedAggregateAndProof - struct 2.6853 ms/op 3.2226 ms/op 0.83
validate gossip signedAggregateAndProof - struct 2.7074 ms/op 3.3276 ms/op 0.81
validate api attestation - struct 1.2791 ms/op 1.5934 ms/op 0.80
validate gossip attestation - struct 1.3968 ms/op 1.6007 ms/op 0.87
pickEth1Vote - no votes 1.0915 ms/op 1.3795 ms/op 0.79
pickEth1Vote - max votes 9.3870 ms/op 12.275 ms/op 0.76
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.2638 ms/op 10.166 ms/op 0.81
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 13.856 ms/op 19.748 ms/op 0.70
pickEth1Vote - Eth1Data fastSerialize value x2048 550.90 us/op 777.65 us/op 0.71
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.9618 ms/op 6.6088 ms/op 0.90
bytes32 toHexString 461.00 ns/op 620.00 ns/op 0.74
bytes32 Buffer.toString(hex) 280.00 ns/op 359.00 ns/op 0.78
bytes32 Buffer.toString(hex) from Uint8Array 419.00 ns/op 575.00 ns/op 0.73
bytes32 Buffer.toString(hex) + 0x 280.00 ns/op 359.00 ns/op 0.78
Object access 1 prop 0.15200 ns/op 0.20200 ns/op 0.75
Map access 1 prop 0.14000 ns/op 0.17700 ns/op 0.79
Object get x1000 6.9050 ns/op 9.5550 ns/op 0.72
Map get x1000 0.50600 ns/op 0.78900 ns/op 0.64
Object set x1000 46.736 ns/op 64.433 ns/op 0.73
Map set x1000 37.027 ns/op 47.571 ns/op 0.78
Return object 10000 times 0.22790 ns/op 0.26970 ns/op 0.85
Throw Error 10000 times 3.7624 us/op 4.6095 us/op 0.82
fastMsgIdFn sha256 / 200 bytes 3.1550 us/op 3.9850 us/op 0.79
fastMsgIdFn h32 xxhash / 200 bytes 267.00 ns/op 362.00 ns/op 0.74
fastMsgIdFn h64 xxhash / 200 bytes 330.00 ns/op 419.00 ns/op 0.79
fastMsgIdFn sha256 / 1000 bytes 10.931 us/op 13.546 us/op 0.81
fastMsgIdFn h32 xxhash / 1000 bytes 389.00 ns/op 503.00 ns/op 0.77
fastMsgIdFn h64 xxhash / 1000 bytes 392.00 ns/op 489.00 ns/op 0.80
fastMsgIdFn sha256 / 10000 bytes 100.14 us/op 121.68 us/op 0.82
fastMsgIdFn h32 xxhash / 10000 bytes 1.8370 us/op 2.2080 us/op 0.83
fastMsgIdFn h64 xxhash / 10000 bytes 1.2780 us/op 1.5380 us/op 0.83
enrSubnets - fastDeserialize 64 bits 1.1740 us/op 1.4820 us/op 0.79
enrSubnets - ssz BitVector 64 bits 409.00 ns/op 489.00 ns/op 0.84
enrSubnets - fastDeserialize 4 bits 163.00 ns/op 196.00 ns/op 0.83
enrSubnets - ssz BitVector 4 bits 408.00 ns/op 495.00 ns/op 0.82
prioritizePeers score -10:0 att 32-0.1 sync 2-0 95.163 us/op 126.29 us/op 0.75
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 119.93 us/op 158.94 us/op 0.75
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 155.78 us/op 169.50 us/op 0.92
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 274.56 us/op 336.05 us/op 0.82
prioritizePeers score 0:0 att 64-1 sync 4-1 322.00 us/op 396.68 us/op 0.81
array of 16000 items push then shift 1.5314 us/op 1.8360 us/op 0.83
LinkedList of 16000 items push then shift 8.6640 ns/op 10.748 ns/op 0.81
array of 16000 items push then pop 55.213 ns/op 69.349 ns/op 0.80
LinkedList of 16000 items push then pop 8.4570 ns/op 10.358 ns/op 0.82
array of 24000 items push then shift 2.3447 us/op 2.6413 us/op 0.89
LinkedList of 24000 items push then shift 8.8490 ns/op 10.288 ns/op 0.86
array of 24000 items push then pop 102.78 ns/op 131.50 ns/op 0.78
LinkedList of 24000 items push then pop 8.4200 ns/op 10.215 ns/op 0.82
intersect bitArray bitLen 8 6.5620 ns/op 7.9810 ns/op 0.82
intersect array and set length 8 54.699 ns/op 63.580 ns/op 0.86
intersect bitArray bitLen 128 31.001 ns/op 39.068 ns/op 0.79
intersect array and set length 128 738.59 ns/op 897.60 ns/op 0.82
bitArray.getTrueBitIndexes() bitLen 128 1.4150 us/op 1.6560 us/op 0.85
bitArray.getTrueBitIndexes() bitLen 248 2.3210 us/op 2.6970 us/op 0.86
bitArray.getTrueBitIndexes() bitLen 512 4.4400 us/op 5.1970 us/op 0.85
Buffer.concat 32 items 1.0160 us/op 1.1030 us/op 0.92
Uint8Array.set 32 items 1.8200 us/op 2.5640 us/op 0.71
transfer serialized Status (84 B) 1.8750 us/op 2.0520 us/op 0.91
copy serialized Status (84 B) 1.5690 us/op 1.6870 us/op 0.93
transfer serialized SignedVoluntaryExit (112 B) 2.0070 us/op 2.0920 us/op 0.96
copy serialized SignedVoluntaryExit (112 B) 1.6370 us/op 2.0250 us/op 0.81
transfer serialized ProposerSlashing (416 B) 2.4180 us/op 2.8810 us/op 0.84
copy serialized ProposerSlashing (416 B) 1.9910 us/op 2.9400 us/op 0.68
transfer serialized Attestation (485 B) 2.3710 us/op 3.2290 us/op 0.73
copy serialized Attestation (485 B) 2.2610 us/op 2.6990 us/op 0.84
transfer serialized AttesterSlashing (33232 B) 2.5790 us/op 2.7220 us/op 0.95
copy serialized AttesterSlashing (33232 B) 5.0490 us/op 6.7110 us/op 0.75
transfer serialized Small SignedBeaconBlock (128000 B) 2.9630 us/op 3.1780 us/op 0.93
copy serialized Small SignedBeaconBlock (128000 B) 12.110 us/op 18.895 us/op 0.64
transfer serialized Avg SignedBeaconBlock (200000 B) 3.3520 us/op 3.2890 us/op 1.02
copy serialized Avg SignedBeaconBlock (200000 B) 16.867 us/op 24.412 us/op 0.69
transfer serialized BlobsSidecar (524380 B) 2.9150 us/op 3.3920 us/op 0.86
copy serialized BlobsSidecar (524380 B) 84.698 us/op 87.581 us/op 0.97
transfer serialized Big SignedBeaconBlock (1000000 B) 2.9810 us/op 3.6580 us/op 0.81
copy serialized Big SignedBeaconBlock (1000000 B) 158.96 us/op 185.05 us/op 0.86
pass gossip attestations to forkchoice per slot 2.0857 ms/op 2.4131 ms/op 0.86
forkChoice updateHead vc 100000 bc 64 eq 0 2.1332 ms/op 2.7444 ms/op 0.78
forkChoice updateHead vc 600000 bc 64 eq 0 13.430 ms/op 15.487 ms/op 0.87
forkChoice updateHead vc 1000000 bc 64 eq 0 22.152 ms/op 26.981 ms/op 0.82
forkChoice updateHead vc 600000 bc 320 eq 0 16.864 ms/op 19.423 ms/op 0.87
forkChoice updateHead vc 600000 bc 1200 eq 0 80.024 ms/op 99.561 ms/op 0.80
forkChoice updateHead vc 600000 bc 64 eq 1000 21.693 ms/op 26.530 ms/op 0.82
forkChoice updateHead vc 600000 bc 64 eq 10000 23.179 ms/op 28.793 ms/op 0.81
forkChoice updateHead vc 600000 bc 64 eq 300000 29.573 ms/op 37.220 ms/op 0.79
computeDeltas 2.9029 ms/op 3.3833 ms/op 0.86
computeProposerBoostScoreFromBalances 381.61 us/op 461.34 us/op 0.83
altair processAttestation - 250000 vs - 7PWei normalcase 1.9469 ms/op 3.0526 ms/op 0.64
altair processAttestation - 250000 vs - 7PWei worstcase 2.9947 ms/op 3.8043 ms/op 0.79
altair processAttestation - setStatus - 1/6 committees join 181.65 us/op 173.76 us/op 1.05
altair processAttestation - setStatus - 1/3 committees join 350.98 us/op 322.47 us/op 1.09
altair processAttestation - setStatus - 1/2 committees join 489.09 us/op 441.22 us/op 1.11
altair processAttestation - setStatus - 2/3 committees join 575.52 us/op 581.78 us/op 0.99
altair processAttestation - setStatus - 4/5 committees join 851.80 us/op 748.55 us/op 1.14
altair processAttestation - setStatus - 100% committees join 958.84 us/op 903.00 us/op 1.06
altair processBlock - 250000 vs - 7PWei normalcase 9.0420 ms/op 10.256 ms/op 0.88
altair processBlock - 250000 vs - 7PWei normalcase hashState 17.546 ms/op 17.803 ms/op 0.99
altair processBlock - 250000 vs - 7PWei worstcase 37.395 ms/op 41.451 ms/op 0.90
altair processBlock - 250000 vs - 7PWei worstcase hashState 56.466 ms/op 68.929 ms/op 0.82
phase0 processBlock - 250000 vs - 7PWei normalcase 1.9903 ms/op 3.0779 ms/op 0.65
phase0 processBlock - 250000 vs - 7PWei worstcase 28.127 ms/op 37.156 ms/op 0.76
altair processEth1Data - 250000 vs - 7PWei normalcase 457.45 us/op 631.10 us/op 0.72
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 7.7090 us/op 14.662 us/op 0.53
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 56.965 us/op 83.848 us/op 0.68
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 15.706 us/op 28.522 us/op 0.55
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 9.7660 us/op 15.691 us/op 0.62
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 135.89 us/op 160.02 us/op 0.85
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.0108 ms/op 1.5842 ms/op 0.64
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.7085 ms/op 1.6811 ms/op 1.02
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.4816 ms/op 2.1995 ms/op 0.67
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.1733 ms/op 4.2154 ms/op 0.75
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.2801 ms/op 3.0623 ms/op 0.74
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.0416 ms/op 6.6233 ms/op 0.76
Tree 40 250000 create 301.46 ms/op 402.11 ms/op 0.75
Tree 40 250000 get(125000) 194.24 ns/op 248.98 ns/op 0.78
Tree 40 250000 set(125000) 819.14 ns/op 1.2224 us/op 0.67
Tree 40 250000 toArray() 16.609 ms/op 23.239 ms/op 0.71
Tree 40 250000 iterate all - toArray() + loop 16.705 ms/op 23.381 ms/op 0.71
Tree 40 250000 iterate all - get(i) 63.546 ms/op 87.545 ms/op 0.73
MutableVector 250000 create 9.2112 ms/op 13.395 ms/op 0.69
MutableVector 250000 get(125000) 6.4270 ns/op 8.8110 ns/op 0.73
MutableVector 250000 set(125000) 231.98 ns/op 352.24 ns/op 0.66
MutableVector 250000 toArray() 2.8007 ms/op 4.7566 ms/op 0.59
MutableVector 250000 iterate all - toArray() + loop 2.9223 ms/op 4.8400 ms/op 0.60
MutableVector 250000 iterate all - get(i) 1.5251 ms/op 2.0256 ms/op 0.75
Array 250000 create 2.6203 ms/op 3.5404 ms/op 0.74
Array 250000 clone - spread 969.44 us/op 1.0557 ms/op 0.92
Array 250000 get(125000) 0.49000 ns/op 0.58200 ns/op 0.84
Array 250000 set(125000) 0.56100 ns/op 0.69900 ns/op 0.80
Array 250000 iterate all - loop 82.194 us/op 119.56 us/op 0.69
effectiveBalanceIncrements clone Uint8Array 300000 20.308 us/op 26.896 us/op 0.76
effectiveBalanceIncrements clone MutableVector 300000 259.00 ns/op 309.00 ns/op 0.84
effectiveBalanceIncrements rw all Uint8Array 300000 176.74 us/op 219.32 us/op 0.81
effectiveBalanceIncrements rw all MutableVector 300000 77.177 ms/op 92.943 ms/op 0.83
phase0 afterProcessEpoch - 250000 vs - 7PWei 110.68 ms/op 127.78 ms/op 0.87
phase0 beforeProcessEpoch - 250000 vs - 7PWei 31.777 ms/op 43.716 ms/op 0.73
altair processEpoch - mainnet_e81889 319.80 ms/op 370.73 ms/op 0.86
mainnet_e81889 - altair beforeProcessEpoch 60.259 ms/op 72.648 ms/op 0.83
mainnet_e81889 - altair processJustificationAndFinalization 13.712 us/op 19.300 us/op 0.71
mainnet_e81889 - altair processInactivityUpdates 5.3850 ms/op 6.3206 ms/op 0.85
mainnet_e81889 - altair processRewardsAndPenalties 51.198 ms/op 78.806 ms/op 0.65
mainnet_e81889 - altair processRegistryUpdates 2.5410 us/op 3.1150 us/op 0.82
mainnet_e81889 - altair processSlashings 488.00 ns/op 875.00 ns/op 0.56
mainnet_e81889 - altair processEth1DataReset 559.00 ns/op 665.00 ns/op 0.84
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2502 ms/op 1.7294 ms/op 0.72
mainnet_e81889 - altair processSlashingsReset 2.9200 us/op 3.5210 us/op 0.83
mainnet_e81889 - altair processRandaoMixesReset 3.6030 us/op 6.1700 us/op 0.58
mainnet_e81889 - altair processHistoricalRootsUpdate 1.0250 us/op 737.00 ns/op 1.39
mainnet_e81889 - altair processParticipationFlagUpdates 1.9580 us/op 3.8180 us/op 0.51
mainnet_e81889 - altair processSyncCommitteeUpdates 541.00 ns/op 1.0430 us/op 0.52
mainnet_e81889 - altair afterProcessEpoch 119.56 ms/op 154.49 ms/op 0.77
phase0 processEpoch - mainnet_e58758 353.61 ms/op 407.33 ms/op 0.87
mainnet_e58758 - phase0 beforeProcessEpoch 130.34 ms/op 156.65 ms/op 0.83
mainnet_e58758 - phase0 processJustificationAndFinalization 14.007 us/op 18.891 us/op 0.74
mainnet_e58758 - phase0 processRewardsAndPenalties 66.771 ms/op 58.110 ms/op 1.15
mainnet_e58758 - phase0 processRegistryUpdates 10.112 us/op 13.438 us/op 0.75
mainnet_e58758 - phase0 processSlashings 524.00 ns/op 887.00 ns/op 0.59
mainnet_e58758 - phase0 processEth1DataReset 448.00 ns/op 940.00 ns/op 0.48
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.2779 ms/op 1.4013 ms/op 0.91
mainnet_e58758 - phase0 processSlashingsReset 1.9100 us/op 2.9490 us/op 0.65
mainnet_e58758 - phase0 processRandaoMixesReset 3.9500 us/op 9.7990 us/op 0.40
mainnet_e58758 - phase0 processHistoricalRootsUpdate 513.00 ns/op 1.0850 us/op 0.47
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.4890 us/op 5.5890 us/op 0.62
mainnet_e58758 - phase0 afterProcessEpoch 94.735 ms/op 125.40 ms/op 0.76
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.1666 ms/op 2.6190 ms/op 0.45
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4243 ms/op 1.9697 ms/op 0.72
altair processInactivityUpdates - 250000 normalcase 23.807 ms/op 30.724 ms/op 0.77
altair processInactivityUpdates - 250000 worstcase 25.992 ms/op 30.561 ms/op 0.85
phase0 processRegistryUpdates - 250000 normalcase 8.0050 us/op 9.8080 us/op 0.82
phase0 processRegistryUpdates - 250000 badcase_full_deposits 295.94 us/op 463.50 us/op 0.64
phase0 processRegistryUpdates - 250000 worstcase 0.5 127.41 ms/op 178.54 ms/op 0.71
altair processRewardsAndPenalties - 250000 normalcase 72.219 ms/op 92.004 ms/op 0.78
altair processRewardsAndPenalties - 250000 worstcase 70.032 ms/op 93.638 ms/op 0.75
phase0 getAttestationDeltas - 250000 normalcase 7.5809 ms/op 10.625 ms/op 0.71
phase0 getAttestationDeltas - 250000 worstcase 7.5053 ms/op 9.7165 ms/op 0.77
phase0 processSlashings - 250000 worstcase 2.2938 ms/op 2.6816 ms/op 0.86
altair processSyncCommitteeUpdates - 250000 143.26 ms/op 186.33 ms/op 0.77
BeaconState.hashTreeRoot - No change 281.00 ns/op 309.00 ns/op 0.91
BeaconState.hashTreeRoot - 1 full validator 49.244 us/op 59.958 us/op 0.82
BeaconState.hashTreeRoot - 32 full validator 479.55 us/op 615.70 us/op 0.78
BeaconState.hashTreeRoot - 512 full validator 4.9708 ms/op 6.9734 ms/op 0.71
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 59.924 us/op 83.981 us/op 0.71
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 821.38 us/op 1.0753 ms/op 0.76
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 10.162 ms/op 13.736 ms/op 0.74
BeaconState.hashTreeRoot - 1 balances 46.087 us/op 65.119 us/op 0.71
BeaconState.hashTreeRoot - 32 balances 407.81 us/op 517.60 us/op 0.79
BeaconState.hashTreeRoot - 512 balances 3.7729 ms/op 5.5757 ms/op 0.68
BeaconState.hashTreeRoot - 250000 balances 70.914 ms/op 92.814 ms/op 0.76
aggregationBits - 2048 els - zipIndexesInBitList 13.821 us/op 17.621 us/op 0.78
regular array get 100000 times 32.152 us/op 40.218 us/op 0.80
wrappedArray get 100000 times 32.164 us/op 39.511 us/op 0.81
arrayWithProxy get 100000 times 14.142 ms/op 17.255 ms/op 0.82
ssz.Root.equals 207.00 ns/op 273.00 ns/op 0.76
byteArrayEquals 208.00 ns/op 277.00 ns/op 0.75
shuffle list - 16384 els 6.7127 ms/op 8.6675 ms/op 0.77
shuffle list - 250000 els 98.596 ms/op 121.36 ms/op 0.81
processSlot - 1 slots 8.3880 us/op 9.4550 us/op 0.89
processSlot - 32 slots 1.2725 ms/op 1.4943 ms/op 0.85
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 55.311 ms/op 59.458 ms/op 0.93
getCommitteeAssignments - req 1 vs - 250000 vc 2.4692 ms/op 3.2875 ms/op 0.75
getCommitteeAssignments - req 100 vs - 250000 vc 3.6130 ms/op 4.7590 ms/op 0.76
getCommitteeAssignments - req 1000 vs - 250000 vc 3.9357 ms/op 5.1554 ms/op 0.76
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.4500 ns/op 5.4900 ns/op 0.81
state getBlockRootAtSlot - 250000 vs - 7PWei 788.59 ns/op 708.43 ns/op 1.11
computeProposers - vc 250000 8.3669 ms/op 10.002 ms/op 0.84
computeEpochShuffling - vc 250000 101.18 ms/op 121.21 ms/op 0.83
getNextSyncCommittee - vc 250000 142.38 ms/op 164.83 ms/op 0.86
computeSigningRoot for AttestationData 12.992 us/op 14.004 us/op 0.93
hash AttestationData serialized data then Buffer.toString(base64) 2.2368 us/op 2.6087 us/op 0.86
toHexString serialized data 1.0297 us/op 1.2046 us/op 0.85
Buffer.toString(base64) 209.32 ns/op 250.66 ns/op 0.84

by benchmarkbot/action

@nflaig
nflaig force-pushed the nflaig/server-graceful-shutdown branch from d2bf447 to 7a1e435 Compare July 22, 2023 12:52
nflaig
nflaig commented Jul 22, 2023
View reviewed changes
Comment thread packages/beacon-node/src/api/rest/base.ts
// Don't log ErrorAborted errors, they happen on node shutdown and are not useful
// Don't log NodeISSyncing errors, they happen very frequently while syncing and the validator polls duties
// Don't log eventstream aborted errors if server instance is being closed on node shutdown
if (err instanceof ErrorAborted || err instanceof NodeIsSyncing || this.status === Status.Closed) return;

@nflaig nflaig Jul 22, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this in #5330 to avoid noisy errors on shutdown, however eventstream aborts are now properly handled and no more error are logged on shutdown.

Since we attempt to gracefully close connections now, it is highly unlikely to log any errors on shutdown since we should only get ErrorAborted which is already checked and not logged. Any error that is not caught here is unexpected and should be logged.

@nflaig
nflaig force-pushed the nflaig/server-graceful-shutdown branch from 3db6051 to 49583c1 Compare July 22, 2023 15:07
@nflaig
nflaig marked this pull request as draft July 22, 2023 15:21
@nflaig
nflaig force-pushed the nflaig/server-graceful-shutdown branch from 49583c1 to 485a996 Compare July 22, 2023 16:23
@nflaig
nflaig force-pushed the nflaig/server-graceful-shutdown branch from 1cd9111 to ff70972 Compare July 23, 2023 11:42
@nflaig
nflaig marked this pull request as ready for review July 23, 2023 11:44
dapplion
dapplion approved these changes Jul 24, 2023
View reviewed changes

@dapplion dapplion left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

wemeetagain
wemeetagain approved these changes Jul 24, 2023
View reviewed changes

@wemeetagain wemeetagain left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃憤

@wemeetagain
wemeetagain merged commit ae9f572 into unstable Jul 24, 2023
@wemeetagain
wemeetagain deleted the nflaig/server-graceful-shutdown branch July 24, 2023 16:59
matthewkeil
matthewkeil reviewed Jul 24, 2023
View reviewed changes

@matthewkeil matthewkeil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post merge review. This is great!!!

@wemeetagain

wemeetagain commented Aug 3, 2023

Copy link
Copy Markdown
Member

馃帀 This PR is included in v1.10.0 馃帀

@nflaig nflaig mentioned this pull request Oct 24, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@matthewkeil matthewkeil matthewkeil left review comments
@wemeetagain wemeetagain wemeetagain approved these changes
+1 more reviewer
@dapplion dapplion dapplion approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@nflaig @wemeetagain @matthewkeil @dapplion