Skip to content

Add option to override preset with custom values from file - #5387

Merged
nflaig merged 1 commit into
unstablefrom
nflaig/override-preset-from-file
Apr 19, 2023
Merged

Add option to override preset with custom values from file#5387
nflaig merged 1 commit into
unstablefrom
nflaig/override-preset-from-file

Conversation

@nflaig

@nflaig nflaig commented Apr 19, 2023

Copy link
Copy Markdown
Member

Motivation

Follow up to #4601 to expose setActivePreset through the CLI and not just at the library level.

Presets are more extensive than runtime configurations, and generally only applicable during compile-time. However, to support various private networks, simulation networks or CI/CD integration tests we require more flexibility.

The main motivation for this PR is to support running Lodestar VC in Obol Network CI/CD integration tests where they use a custom configuration that requires to override preset values such as SLOTS_PER_EPOCH and EPOCHS_PER_SYNC_COMMITTEE_PERIOD.

Description

Adds CLI flag --presetFile which invokes setActivePreset with custom values from file to override values of the active preset.

Usage

preset.yaml

SLOTS_PER_EPOCH: 16
EPOCHS_PER_SYNC_COMMITTEE_PERIOD: 2
lodestar beacon --presetFile "./preset.yaml"

Adds CLI flag `--presetFile` which invokes `setActivePreset` with
custom values from file to override values of the active preset.
@nflaig
nflaig requested a review from a team as a code owner April 19, 2023 13:21
},

presetFile: {
hidden: true,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃憤

@github-actions

Copy link
Copy Markdown
Contributor

Performance Report

鉁旓笍 no performance regression detected

Full benchmark results
Benchmark suite Current: 3177a89 Previous: 0488388 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 861.18 us/op 684.10 us/op 1.26
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 47.370 us/op 45.818 us/op 1.03
BLS verify - blst-native 1.2256 ms/op 1.2301 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 2.5028 ms/op 2.5005 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 5.3710 ms/op 5.3847 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 19.481 ms/op 19.287 ms/op 1.01
BLS aggregatePubkeys 32 - blst-native 26.335 us/op 25.783 us/op 1.02
BLS aggregatePubkeys 128 - blst-native 102.45 us/op 100.36 us/op 1.02
getAttestationsForBlock 55.953 ms/op 58.234 ms/op 0.96
isKnown best case - 1 super set check 257.00 ns/op 255.00 ns/op 1.01
isKnown normal case - 2 super set checks 249.00 ns/op 246.00 ns/op 1.01
isKnown worse case - 16 super set checks 246.00 ns/op 245.00 ns/op 1.00
CheckpointStateCache - add get delete 5.5080 us/op 5.3330 us/op 1.03
validate gossip signedAggregateAndProof - struct 2.8293 ms/op 2.8063 ms/op 1.01
validate gossip attestation - struct 1.3515 ms/op 1.3399 ms/op 1.01
pickEth1Vote - no votes 1.2941 ms/op 1.2577 ms/op 1.03
pickEth1Vote - max votes 9.5201 ms/op 8.6434 ms/op 1.10
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.8141 ms/op 8.6057 ms/op 1.02
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 16.263 ms/op 18.170 ms/op 0.90
pickEth1Vote - Eth1Data fastSerialize value x2048 756.72 us/op 657.12 us/op 1.15
pickEth1Vote - Eth1Data fastSerialize tree x2048 8.3425 ms/op 5.9346 ms/op 1.41
bytes32 toHexString 539.00 ns/op 468.00 ns/op 1.15
bytes32 Buffer.toString(hex) 383.00 ns/op 332.00 ns/op 1.15
bytes32 Buffer.toString(hex) from Uint8Array 591.00 ns/op 548.00 ns/op 1.08
bytes32 Buffer.toString(hex) + 0x 382.00 ns/op 339.00 ns/op 1.13
Object access 1 prop 0.17000 ns/op 0.15800 ns/op 1.08
Map access 1 prop 0.16300 ns/op 0.15800 ns/op 1.03
Object get x1000 6.7120 ns/op 6.8310 ns/op 0.98
Map get x1000 0.65800 ns/op 0.58600 ns/op 1.12
Object set x1000 62.164 ns/op 50.996 ns/op 1.22
Map set x1000 50.569 ns/op 42.327 ns/op 1.19
Return object 10000 times 0.24710 ns/op 0.23520 ns/op 1.05
Throw Error 10000 times 4.3064 us/op 4.1825 us/op 1.03
fastMsgIdFn sha256 / 200 bytes 3.6680 us/op 3.4890 us/op 1.05
fastMsgIdFn h32 xxhash / 200 bytes 313.00 ns/op 286.00 ns/op 1.09
fastMsgIdFn h64 xxhash / 200 bytes 477.00 ns/op 411.00 ns/op 1.16
fastMsgIdFn sha256 / 1000 bytes 12.383 us/op 11.632 us/op 1.06
fastMsgIdFn h32 xxhash / 1000 bytes 444.00 ns/op 403.00 ns/op 1.10
fastMsgIdFn h64 xxhash / 1000 bytes 504.00 ns/op 485.00 ns/op 1.04
fastMsgIdFn sha256 / 10000 bytes 106.17 us/op 104.77 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 2.0480 us/op 1.9430 us/op 1.05
fastMsgIdFn h64 xxhash / 10000 bytes 1.4370 us/op 1.3790 us/op 1.04
enrSubnets - fastDeserialize 64 bits 1.4800 us/op 1.3000 us/op 1.14
enrSubnets - ssz BitVector 64 bits 546.00 ns/op 487.00 ns/op 1.12
enrSubnets - fastDeserialize 4 bits 184.00 ns/op 175.00 ns/op 1.05
enrSubnets - ssz BitVector 4 bits 533.00 ns/op 481.00 ns/op 1.11
prioritizePeers score -10:0 att 32-0.1 sync 2-0 113.41 us/op 104.42 us/op 1.09
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 153.58 us/op 139.31 us/op 1.10
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 196.15 us/op 172.10 us/op 1.14
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 327.13 us/op 309.87 us/op 1.06
prioritizePeers score 0:0 att 64-1 sync 4-1 381.72 us/op 395.45 us/op 0.97
array of 16000 items push then shift 1.7176 us/op 1.7312 us/op 0.99
LinkedList of 16000 items push then shift 9.0290 ns/op 9.1640 ns/op 0.99
array of 16000 items push then pop 111.16 ns/op 103.97 ns/op 1.07
LinkedList of 16000 items push then pop 8.6700 ns/op 8.8440 ns/op 0.98
array of 24000 items push then shift 2.3916 us/op 2.4702 us/op 0.97
LinkedList of 24000 items push then shift 8.9430 ns/op 9.9170 ns/op 0.90
array of 24000 items push then pop 82.666 ns/op 80.720 ns/op 1.02
LinkedList of 24000 items push then pop 8.6810 ns/op 8.9020 ns/op 0.98
intersect bitArray bitLen 8 13.182 ns/op 13.643 ns/op 0.97
intersect array and set length 8 79.255 ns/op 80.631 ns/op 0.98
intersect bitArray bitLen 128 44.251 ns/op 46.155 ns/op 0.96
intersect array and set length 128 1.0741 us/op 1.0996 us/op 0.98
Buffer.concat 32 items 2.9870 us/op 2.6980 us/op 1.11
Uint8Array.set 32 items 2.9310 us/op 2.3990 us/op 1.22
pass gossip attestations to forkchoice per slot 3.0304 ms/op 3.4207 ms/op 0.89
computeDeltas 3.1236 ms/op 3.0077 ms/op 1.04
computeProposerBoostScoreFromBalances 1.8133 ms/op 1.8160 ms/op 1.00
altair processAttestation - 250000 vs - 7PWei normalcase 2.5326 ms/op 2.3409 ms/op 1.08
altair processAttestation - 250000 vs - 7PWei worstcase 3.3627 ms/op 3.3511 ms/op 1.00
altair processAttestation - setStatus - 1/6 committees join 141.82 us/op 141.86 us/op 1.00
altair processAttestation - setStatus - 1/3 committees join 282.62 us/op 285.53 us/op 0.99
altair processAttestation - setStatus - 1/2 committees join 375.05 us/op 387.71 us/op 0.97
altair processAttestation - setStatus - 2/3 committees join 480.36 us/op 472.30 us/op 1.02
altair processAttestation - setStatus - 4/5 committees join 668.33 us/op 662.64 us/op 1.01
altair processAttestation - setStatus - 100% committees join 801.69 us/op 790.95 us/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase 18.381 ms/op 18.925 ms/op 0.97
altair processBlock - 250000 vs - 7PWei normalcase hashState 25.865 ms/op 27.072 ms/op 0.96
altair processBlock - 250000 vs - 7PWei worstcase 50.968 ms/op 57.712 ms/op 0.88
altair processBlock - 250000 vs - 7PWei worstcase hashState 70.433 ms/op 75.360 ms/op 0.93
phase0 processBlock - 250000 vs - 7PWei normalcase 2.1585 ms/op 2.4976 ms/op 0.86
phase0 processBlock - 250000 vs - 7PWei worstcase 31.894 ms/op 28.264 ms/op 1.13
altair processEth1Data - 250000 vs - 7PWei normalcase 470.13 us/op 496.08 us/op 0.95
vc - 250000 eb 1 eth1 1 we 0 wn 0 - smpl 15 7.3270 us/op 9.6390 us/op 0.76
vc - 250000 eb 0.95 eth1 0.1 we 0.05 wn 0 - smpl 219 25.916 us/op 25.990 us/op 1.00
vc - 250000 eb 0.95 eth1 0.3 we 0.05 wn 0 - smpl 42 9.8570 us/op 11.440 us/op 0.86
vc - 250000 eb 0.95 eth1 0.7 we 0.05 wn 0 - smpl 18 7.6780 us/op 9.1820 us/op 0.84
vc - 250000 eb 0.1 eth1 0.1 we 0 wn 0 - smpl 1020 98.093 us/op 113.64 us/op 0.86
vc - 250000 eb 0.03 eth1 0.03 we 0 wn 0 - smpl 11777 639.75 us/op 1.2595 ms/op 0.51
vc - 250000 eb 0.01 eth1 0.01 we 0 wn 0 - smpl 16384 913.13 us/op 1.1484 ms/op 0.80
vc - 250000 eb 0 eth1 0 we 0 wn 0 - smpl 16384 894.53 us/op 1.1017 ms/op 0.81
vc - 250000 eb 0 eth1 0 we 0 wn 0 nocache - smpl 16384 2.3899 ms/op 2.3799 ms/op 1.00
vc - 250000 eb 0 eth1 1 we 0 wn 0 - smpl 16384 1.5347 ms/op 2.0397 ms/op 0.75
vc - 250000 eb 0 eth1 1 we 0 wn 0 nocache - smpl 16384 3.7898 ms/op 4.2399 ms/op 0.89
Tree 40 250000 create 372.88 ms/op 373.32 ms/op 1.00
Tree 40 250000 get(125000) 198.24 ns/op 189.79 ns/op 1.04
Tree 40 250000 set(125000) 994.59 ns/op 1.0752 us/op 0.92
Tree 40 250000 toArray() 21.419 ms/op 18.838 ms/op 1.14
Tree 40 250000 iterate all - toArray() + loop 22.131 ms/op 22.012 ms/op 1.01
Tree 40 250000 iterate all - get(i) 74.680 ms/op 73.283 ms/op 1.02
MutableVector 250000 create 12.752 ms/op 11.669 ms/op 1.09
MutableVector 250000 get(125000) 6.4710 ns/op 6.4970 ns/op 1.00
MutableVector 250000 set(125000) 275.62 ns/op 275.16 ns/op 1.00
MutableVector 250000 toArray() 2.9215 ms/op 3.5817 ms/op 0.82
MutableVector 250000 iterate all - toArray() + loop 3.4842 ms/op 3.2801 ms/op 1.06
MutableVector 250000 iterate all - get(i) 1.5608 ms/op 1.5406 ms/op 1.01
Array 250000 create 2.7867 ms/op 2.9054 ms/op 0.96
Array 250000 clone - spread 1.1342 ms/op 1.2070 ms/op 0.94
Array 250000 get(125000) 0.60600 ns/op 0.59300 ns/op 1.02
Array 250000 set(125000) 0.63800 ns/op 0.72100 ns/op 0.88
Array 250000 iterate all - loop 81.999 us/op 104.47 us/op 0.78
effectiveBalanceIncrements clone Uint8Array 300000 28.374 us/op 34.344 us/op 0.83
effectiveBalanceIncrements clone MutableVector 300000 344.00 ns/op 379.00 ns/op 0.91
effectiveBalanceIncrements rw all Uint8Array 300000 165.79 us/op 173.19 us/op 0.96
effectiveBalanceIncrements rw all MutableVector 300000 85.408 ms/op 85.619 ms/op 1.00
phase0 afterProcessEpoch - 250000 vs - 7PWei 115.87 ms/op 120.64 ms/op 0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei 41.069 ms/op 36.469 ms/op 1.13
altair processEpoch - mainnet_e81889 310.52 ms/op 355.83 ms/op 0.87
mainnet_e81889 - altair beforeProcessEpoch 53.567 ms/op 73.129 ms/op 0.73
mainnet_e81889 - altair processJustificationAndFinalization 16.826 us/op 19.784 us/op 0.85
mainnet_e81889 - altair processInactivityUpdates 6.0907 ms/op 7.2058 ms/op 0.85
mainnet_e81889 - altair processRewardsAndPenalties 70.247 ms/op 62.120 ms/op 1.13
mainnet_e81889 - altair processRegistryUpdates 2.7280 us/op 3.1660 us/op 0.86
mainnet_e81889 - altair processSlashings 545.00 ns/op 624.00 ns/op 0.87
mainnet_e81889 - altair processEth1DataReset 562.00 ns/op 1.0340 us/op 0.54
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2559 ms/op 1.8792 ms/op 0.67
mainnet_e81889 - altair processSlashingsReset 4.9380 us/op 8.2910 us/op 0.60
mainnet_e81889 - altair processRandaoMixesReset 5.4950 us/op 7.7340 us/op 0.71
mainnet_e81889 - altair processHistoricalRootsUpdate 1.1830 us/op 3.1600 us/op 0.37
mainnet_e81889 - altair processParticipationFlagUpdates 5.3840 us/op 4.0250 us/op 1.34
mainnet_e81889 - altair processSyncCommitteeUpdates 946.00 ns/op 621.00 ns/op 1.52
mainnet_e81889 - altair afterProcessEpoch 128.34 ms/op 129.61 ms/op 0.99
phase0 processEpoch - mainnet_e58758 351.41 ms/op 390.17 ms/op 0.90
mainnet_e58758 - phase0 beforeProcessEpoch 134.01 ms/op 143.07 ms/op 0.94
mainnet_e58758 - phase0 processJustificationAndFinalization 20.676 us/op 23.176 us/op 0.89
mainnet_e58758 - phase0 processRewardsAndPenalties 52.511 ms/op 70.838 ms/op 0.74
mainnet_e58758 - phase0 processRegistryUpdates 13.719 us/op 9.7770 us/op 1.40
mainnet_e58758 - phase0 processSlashings 1.3600 us/op 715.00 ns/op 1.90
mainnet_e58758 - phase0 processEth1DataReset 1.1310 us/op 829.00 ns/op 1.36
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.8181 ms/op 1.4953 ms/op 1.22
mainnet_e58758 - phase0 processSlashingsReset 8.2100 us/op 5.2100 us/op 1.58
mainnet_e58758 - phase0 processRandaoMixesReset 10.685 us/op 6.4450 us/op 1.66
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.8070 us/op 808.00 ns/op 2.24
mainnet_e58758 - phase0 processParticipationRecordUpdates 12.839 us/op 4.2620 us/op 3.01
mainnet_e58758 - phase0 afterProcessEpoch 110.47 ms/op 104.85 ms/op 1.05
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.0944 ms/op 1.4448 ms/op 1.45
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.7419 ms/op 1.9220 ms/op 1.95
altair processInactivityUpdates - 250000 normalcase 31.698 ms/op 27.960 ms/op 1.13
altair processInactivityUpdates - 250000 worstcase 35.538 ms/op 28.511 ms/op 1.25
phase0 processRegistryUpdates - 250000 normalcase 12.737 us/op 7.3590 us/op 1.73
phase0 processRegistryUpdates - 250000 badcase_full_deposits 324.75 us/op 322.81 us/op 1.01
phase0 processRegistryUpdates - 250000 worstcase 0.5 136.51 ms/op 137.99 ms/op 0.99
altair processRewardsAndPenalties - 250000 normalcase 81.610 ms/op 72.452 ms/op 1.13
altair processRewardsAndPenalties - 250000 worstcase 97.605 ms/op 69.613 ms/op 1.40
phase0 getAttestationDeltas - 250000 normalcase 8.8642 ms/op 8.0396 ms/op 1.10
phase0 getAttestationDeltas - 250000 worstcase 7.3168 ms/op 6.9520 ms/op 1.05
phase0 processSlashings - 250000 worstcase 3.6033 ms/op 3.6937 ms/op 0.98
altair processSyncCommitteeUpdates - 250000 188.28 ms/op 192.64 ms/op 0.98
BeaconState.hashTreeRoot - No change 280.00 ns/op 353.00 ns/op 0.79
BeaconState.hashTreeRoot - 1 full validator 59.346 us/op 57.874 us/op 1.03
BeaconState.hashTreeRoot - 32 full validator 699.72 us/op 574.40 us/op 1.22
BeaconState.hashTreeRoot - 512 full validator 7.6823 ms/op 5.1534 ms/op 1.49
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 80.490 us/op 68.727 us/op 1.17
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2469 ms/op 955.80 us/op 1.30
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.666 ms/op 12.190 ms/op 1.37
BeaconState.hashTreeRoot - 1 balances 72.764 us/op 54.696 us/op 1.33
BeaconState.hashTreeRoot - 32 balances 497.48 us/op 471.04 us/op 1.06
BeaconState.hashTreeRoot - 512 balances 5.9649 ms/op 4.8112 ms/op 1.24
BeaconState.hashTreeRoot - 250000 balances 88.348 ms/op 84.201 ms/op 1.05
aggregationBits - 2048 els - zipIndexesInBitList 28.569 us/op 16.979 us/op 1.68
regular array get 100000 times 53.804 us/op 33.368 us/op 1.61
wrappedArray get 100000 times 37.662 us/op 33.399 us/op 1.13
arrayWithProxy get 100000 times 17.665 ms/op 15.715 ms/op 1.12
ssz.Root.equals 728.00 ns/op 562.00 ns/op 1.30
byteArrayEquals 738.00 ns/op 560.00 ns/op 1.32
shuffle list - 16384 els 9.2487 ms/op 7.0131 ms/op 1.32
shuffle list - 250000 els 119.81 ms/op 101.94 ms/op 1.18
processSlot - 1 slots 11.645 us/op 9.7650 us/op 1.19
processSlot - 32 slots 1.6050 ms/op 1.3646 ms/op 1.18
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 48.832 ms/op 38.669 ms/op 1.26
getCommitteeAssignments - req 1 vs - 250000 vc 3.5876 ms/op 3.0974 ms/op 1.16
getCommitteeAssignments - req 100 vs - 250000 vc 5.0113 ms/op 4.1906 ms/op 1.20
getCommitteeAssignments - req 1000 vs - 250000 vc 5.3321 ms/op 4.6461 ms/op 1.15
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.7200 ns/op 5.1100 ns/op 1.12
state getBlockRootAtSlot - 250000 vs - 7PWei 763.61 ns/op 1.0149 us/op 0.75
computeProposers - vc 250000 12.871 ms/op 11.333 ms/op 1.14
computeEpochShuffling - vc 250000 122.20 ms/op 105.23 ms/op 1.16
getNextSyncCommittee - vc 250000 227.94 ms/op 187.17 ms/op 1.22
computeSigningRoot for AttestationData 16.202 us/op 13.838 us/op 1.17
hash AttestationData serialized data then Buffer.toString(base64) 2.7158 us/op 2.5343 us/op 1.07
toHexString serialized data 2.0426 us/op 1.1348 us/op 1.80
Buffer.toString(base64) 400.68 ns/op 353.23 ns/op 1.13

by benchmarkbot/action

@nflaig
nflaig merged commit 35fe214 into unstable Apr 19, 2023
@nflaig
nflaig deleted the nflaig/override-preset-from-file branch April 19, 2023 17:53
@wemeetagain

Copy link
Copy Markdown
Member

馃帀 This PR is included in v1.8.0 馃帀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants