Skip to content

Add BlobMetrics #7622

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fab-10 merged 6 commits into from
Sep 19, 2024
Merged

Add BlobMetrics #7622

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3abcc01
Add BlobMetrics
7suyash7 Sep 14, 2024
d89e8ee
refactor
7suyash7 Sep 17, 2024
f3ecbf0
remove unused blob_storage
7suyash7 Sep 18, 2024
1a89763
add .size() to BlobCache
7suyash7 Sep 19, 2024
99ecf94
Add to Changelog
7suyash7 Sep 19, 2024
5ba722f
Merge branch 'main' into TransactionPoolMetrics
fab-10 Sep 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion ...eum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ public class TransactionPool implements BlockAddedObserver {
private final EthScheduler.OrderedProcessor<BlockAddedEvent> blockAddedEventOrderedProcessor;
private final Map<VersionedHash, BlobsWithCommitments.BlobQuad> mapOfBlobsInTransactionPool =
new HashMap<>();
private final ConcurrentHashMap<VersionedHash, BlobsWithCommitments.BlobQuad> blobCacheTracker =
new ConcurrentHashMap<>();

@fab-10 fab-10 Sep 17, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this for? it seems to have no real usage

@7suyash7 7suyash7 Sep 18, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's being used to maintain the count of blobs in the cache...

@fab-10 fab-10 Sep 18, 2024

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to add a size method to BlobCache, it is simpler and avoid keeping this extra map

@7suyash7 7suyash7 Sep 19, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I was trying to avoid doing that... I have added it now and removed the extra map


public TransactionPool(
final Supplier<PendingTransactions> pendingTransactionsSupplier,
Expand All @@ -129,6 +131,7 @@ public TransactionPool(
this.blockAddedEventOrderedProcessor =
ethContext.getScheduler().createOrderedProcessor(this::processBlockAddedEvent);
this.cacheForBlobsOfTransactionsAddedToABlock = blobCache;
initializeBlobMetrics();
initLogForReplay();
subscribePendingTransactions(this::mapBlobsOnTransactionAdded);
subscribeDroppedTransactions(this::unmapBlobsOnTransactionDropped);
Expand Down Expand Up @@ -639,6 +642,7 @@ public CompletableFuture<Void> setDisabled() {
pendingTransactionsListenersProxy.unsubscribe();
final PendingTransactions pendingTransactionsToSave = pendingTransactions;
pendingTransactions = new DisabledPendingTransactions();
clearBlobCacheTracker();
return saveRestoreManager
.saveToDisk(pendingTransactionsToSave)
.exceptionally(
Expand Down Expand Up @@ -671,13 +675,20 @@ private void unmapBlobsOnTransactionDropped(
}
final List<BlobsWithCommitments.BlobQuad> blobQuads =
maybeBlobsWithCommitments.get().getBlobQuads();
blobQuads.forEach(bq -> mapOfBlobsInTransactionPool.remove(bq.versionedHash()));
blobQuads.forEach(
bq -> {
mapOfBlobsInTransactionPool.remove(bq.versionedHash());
removeBlobFromCache(bq.versionedHash());
});
}

public BlobsWithCommitments.BlobQuad getBlobQuad(final VersionedHash vh) {
BlobsWithCommitments.BlobQuad blobQuad = mapOfBlobsInTransactionPool.get(vh);
if (blobQuad == null) {
blobQuad = cacheForBlobsOfTransactionsAddedToABlock.get(vh);
if (blobQuad != null) {
blobCacheTracker.putIfAbsent(vh, blobQuad);
}
}
return blobQuad;
}
Expand All @@ -686,6 +697,27 @@ public boolean isEnabled() {
return isPoolEnabled.get();
}

public int getBlobCacheSize() {
return blobCacheTracker.size();
}

public int getBlobMapSize() {
return mapOfBlobsInTransactionPool.size();
}

private void removeBlobFromCache(final VersionedHash vh) {
blobCacheTracker.remove(vh);
}

private void clearBlobCacheTracker() {
blobCacheTracker.clear();
}

private void initializeBlobMetrics() {
metrics.createBlobCacheSizeMetric(this::getBlobCacheSize);
metrics.createBlobMapSizeMetric(this::getBlobMapSize);
}

class PendingTransactionsListenersProxy {
private final Subscribers<PendingTransactionAddedListener> onAddedListeners =
Subscribers.create();
Expand Down
17 changes: 17 additions & 0 deletions .../src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPoolMetrics.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import java.util.HashMap;
import java.util.Map;
import java.util.function.DoubleSupplier;
import java.util.function.IntSupplier;

import org.apache.commons.lang3.tuple.Pair;
import org.slf4j.Logger;
Expand Down Expand Up @@ -285,4 +286,20 @@ private String location(final boolean receivedFromLocalSource) {
private String priority(final boolean hasPriority) {
return hasPriority ? "yes" : "no";
}

public void createBlobCacheSizeMetric(final IntSupplier sizeSupplier) {
metricsSystem.createIntegerGauge(
BesuMetricCategory.TRANSACTION_POOL,
"blob_cache_size",
"Current size of the blob cache",
sizeSupplier);
}

public void createBlobMapSizeMetric(final IntSupplier sizeSupplier) {
metricsSystem.createIntegerGauge(
BesuMetricCategory.TRANSACTION_POOL,
"blob_map_size",
"Current size of the blob map",
sizeSupplier);
}
}
4 changes: 3 additions & 1 deletion metrics/core/src/main/java/org/hyperledger/besu/metrics/BesuMetricCategory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,9 @@ public enum BesuMetricCategory implements MetricCategory {
/** Stratum besu metric category. */
STRATUM("stratum"),
/** Block processing besu metric category. */
BLOCK_PROCESSING("block_processing");
BLOCK_PROCESSING("block_processing"),
/** Blob storage besu metric category. */
BLOB_STORAGE("blob_storage");
Comment thread
7suyash7 marked this conversation as resolved.
Outdated

private static final Optional<String> BESU_PREFIX = Optional.of("besu_");

Expand Down