Add chain data pruning experimental feature #4686
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| /* | ||
| * Copyright Hyperledger Besu Contributors. | ||
| * | ||
| * Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on | ||
| * an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations under the License. | ||
| * | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| * | ||
| */ | ||
| package org.hyperledger.besu.cli.options.unstable; | ||
|
|
||
| import picocli.CommandLine; | ||
|
|
||
| public class ChainDataPruningOptions { | ||
|
|
||
| public static final long DEFAULT_CHAIN_DATA_PRUNING_MIN_BLOCKS_RETAINED = 50400; | ||
|
Contributor
There was a problem hiding this comment. Is there a way to avoid this magic value? If syncing takes longer than a week then this will fail?
Contributor
Author
There was a problem hiding this comment. Yes, ideally if the pruner has the syncing context to know the current pivot block and if the syncing has finished, then it can avoid pruning beyond the pivot block before syncing is completed. Do you think we should do it in a separate ticket?
Contributor
There was a problem hiding this comment. I think we should do this in another ticket.
Contributor
There was a problem hiding this comment. Rename to MAX_CHAIN_DATA_PRUNING_MIN_BLOCKS_RETAINED? This not just the default but also the maximum number of blocks that will be pruned.
Contributor
Author
There was a problem hiding this comment. This is actually the minimum value for the blocks to retain. I've also changed it to be much smaller. |
||
| public static final long DEFAULT_CHAIN_DATA_PRUNING_FREQUENCY = 256; | ||
|
|
||
| @CommandLine.Option( | ||
| hidden = true, | ||
| names = {"--Xchain-data-pruning-enabled"}, | ||
|
wcgcyx marked this conversation as resolved.
Outdated
|
||
| description = | ||
| "Enable the chain pruner to actively prune old chain data (default: ${DEFAULT-VALUE})") | ||
| private final Boolean chainDataPruningEnabled = Boolean.FALSE; | ||
|
|
||
| @CommandLine.Option( | ||
| hidden = true, | ||
| names = {"--Xchain-data-pruning-blocks-retained"}, | ||
| description = | ||
| "The number of recent blocks for which to keep the chain data. Must be >= 50400 (default: ${DEFAULT-VALUE})") | ||
|
wcgcyx marked this conversation as resolved.
Outdated
|
||
| private final Long chainDataPruningBlocksRetained = | ||
| DEFAULT_CHAIN_DATA_PRUNING_MIN_BLOCKS_RETAINED; | ||
|
|
||
| @CommandLine.Option( | ||
| hidden = true, | ||
| names = {"--Xchain-data-pruning-frequency"}, | ||
| description = | ||
| "The number of blocks added to the chain between two pruning operations. Must be non-negative (default: ${DEFAULT-VALUE})") | ||
| private final Long chainDataPruningBlocksFrequency = DEFAULT_CHAIN_DATA_PRUNING_FREQUENCY; | ||
|
|
||
| public static ChainDataPruningOptions create() { | ||
| return new ChainDataPruningOptions(); | ||
| } | ||
|
|
||
| public Boolean getChainDataPruningEnabled() { | ||
| return chainDataPruningEnabled; | ||
| } | ||
|
|
||
| public Long getChainDataPruningBlocksRetained() { | ||
| return chainDataPruningBlocksRetained; | ||
| } | ||
|
|
||
| public Long getChainDataPruningBlocksFrequency() { | ||
|
Contributor
There was a problem hiding this comment. Is there a reason this type doesn't match the PositiveNumber type above?
Contributor
Author
There was a problem hiding this comment. Converting and returning long makes it easy to compare with block number (also in long) in use of outside. Using Positive number is to ensure user can only supply positive frequency. |
||
| return chainDataPruningBlocksFrequency; | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -35,6 +35,7 @@ | |
| import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcResponse; | ||
| import org.hyperledger.besu.ethereum.api.jsonrpc.internal.response.JsonRpcSuccessResponse; | ||
| import org.hyperledger.besu.ethereum.api.jsonrpc.internal.results.EngineUpdateForkchoiceResult; | ||
| import org.hyperledger.besu.ethereum.chain.ChainDataPruner; | ||
| import org.hyperledger.besu.ethereum.core.BlockHeader; | ||
|
|
||
| import java.util.Optional; | ||
|
|
@@ -116,7 +117,8 @@ public JsonRpcResponse syncResponse(final JsonRpcRequestContext requestContext) | |
| } | ||
|
|
||
| // TODO: post-merge cleanup, this should be unnecessary after merge | ||
| if (!mergeCoordinator.latestValidAncestorDescendsFromTerminal(newHead.get()) | ||
| && !ChainDataPruner.isPruningEnabled()) { | ||
|
Contributor
There was a problem hiding this comment. Would be nicer to drive this based on the chain data pruning enabled flag rather than setting and using a static global field
Contributor
Author
There was a problem hiding this comment. Yea, I tried that a bit but it seems like that would involve bringing besu command into the protocol context. I consider it to be a temporary solution until #4703 is merged. Then, we could just do a cleanup to get rid of this global field completely.
Contributor
There was a problem hiding this comment. We shouldn't need to bring the entire Besu command into the protocol context. At most, the chain pruning config values would need to be passed through. A compromise could be to make the ChainPruner a singleton instead.
Contributor
Author
There was a problem hiding this comment. Actually, It seems cleaner to add a flag in merge context instead, following what @gfukushima did in #4735
Contributor
Author
There was a problem hiding this comment. updated to have a temporary flag in protocol context instead. Global variable gets removed. |
||
| logForkchoiceUpdatedCall(INVALID, forkChoice); | ||
| return new JsonRpcSuccessResponse( | ||
| requestId, | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.