Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable txpool when not in sync #6302

Merged
merged 19 commits into from
Jan 10, 2024

Conversation

fab-10
Copy link
Contributor

@fab-10 fab-10 commented Dec 14, 2023

PR description

With this PR the transaction pool factory is now aware if Besu is in sync or not, using the events triggered by the SyncState, and will disable the txpool as soon as the node is out of sync, so avoiding validating txs while not necessary.
This PR also implements a better way to process the onBlockAdded event in the txpool, avoiding the possibility of a tight loop that could result in high cpu usage.

Fixed Issue(s)

fixes #6001

Copy link

github-actions bot commented Dec 14, 2023

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.
  • I thought about the changelog and included a changelog update if required.
  • If my PR includes database changes (e.g. KeyValueSegmentIdentifier) I have thought about compatibility and performed forwards and backwards compatibility tests

@fab-10 fab-10 force-pushed the disable-txpool-during-fullsync branch 5 times, most recently from 1ec9dec to f63e65d Compare December 19, 2023 14:59
@fab-10 fab-10 force-pushed the disable-txpool-during-fullsync branch from f63e65d to 2509e05 Compare December 19, 2023 17:05
@fab-10 fab-10 changed the title Disable txpool during fullsync Disable txpool when not in sync Dec 21, 2023
# Conflicts:
#	ethereum/eth/src/main/java/org/hyperledger/besu/ethereum/eth/transactions/TransactionPool.java
Signed-off-by: Fabio Di Fabio <[email protected]>
@fab-10 fab-10 marked this pull request as ready for review December 21, 2023 13:51
@ahamlat
Copy link
Contributor

ahamlat commented Jan 2, 2024

@fab-10 I tested on a mainnet forest archive sync'ing node, and it worked. I don't see anymore the validation of remote transactions. I see the deactivation in the logs :

{"@timestamp":"2024-01-02T08:52:02,252","level":"INFO","thread":"nioEventLoopGroup-3-6","class":"TransactionPoolFactory","message":"Node out of sync, disabling transaction handling","throwable":""}

I also confirmed by checking the metrics :
image
image

However, I still see that we are announcing new transactions hashes :

image

@fab-10
Copy link
Contributor Author

fab-10 commented Jan 3, 2024

@ahamlat about

I still see that we are announcing new transactions hashes

what you see are the incoming messages that we still receive, but are just ignored and not processed, you should check the outbound panel to verify that we are not sending them.

@ahamlat
Copy link
Contributor

ahamlat commented Jan 4, 2024

Right, I was looking to the inbound messages. When checking outbound messages on the transaction pool, we can see that we have no traffic, which validates this PR.

image

@fab-10 fab-10 enabled auto-merge (squash) January 10, 2024 13:24
@fab-10 fab-10 merged commit 0b4b815 into hyperledger:main Jan 10, 2024
18 checks passed
@fab-10 fab-10 deleted the disable-txpool-during-fullsync branch January 10, 2024 14:01
fab-10 added a commit to Consensys/linea-besu that referenced this pull request Jan 12, 2024
* mark deleted slot during clear storage step (hyperledger#6305)

Signed-off-by: Karim Taam <[email protected]>
Co-authored-by: garyschulte <[email protected]>

* made directory structure of tests match source; fixed one typo (hyperledger#6337)

Signed-off-by: Sally MacFarlane <[email protected]>

* migrate controller tests to junit 5 (hyperledger#6338)

Signed-off-by: Sally MacFarlane <[email protected]>

* add new forkids for testnets, update forkid test to Junit5, no longer need named network specific trusted setups (hyperledger#6322)

Signed-off-by: jflo <[email protected]>

* Fix trielog shipping issue during self destruct (hyperledger#6340)

* fix trielog shipping issue

Signed-off-by: Karim Taam <[email protected]>

* bump gradle properties version and adjust changelog to match release (hyperledger#6347)

Signed-off-by: garyschulte <[email protected]>

* finalized cancun spec (hyperledger#6351)

* finalized cancun spec

Signed-off-by: jflo <[email protected]>

* finalized cancun spec

Signed-off-by: jflo <[email protected]>

---------

Signed-off-by: jflo <[email protected]>

* Optimize RocksDB WAL file (hyperledger#6328)


Signed-off-by: Fabio Di Fabio <[email protected]>

* Make RPC reason settable, pass execution failure reason in RPC error message (hyperledger#6343)

* Make RPC reason settable, pass execution failure reason in RPC error message

Signed-off-by: Matthew Whitehead <[email protected]>

* Update unit tests

Signed-off-by: Matthew Whitehead <[email protected]>

* Update tests

Signed-off-by: Matthew Whitehead <[email protected]>

* Update change log

Signed-off-by: Matthew Whitehead <[email protected]>

* Update integration tests

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>

* TestWatcher junit5 (hyperledger#6339)

* TestWatcher junit5
* add test class and method name to context
* moved the testwatcher junit5 function to a new junit5 superclass
* one qbft test to junit5 superclass

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: Sally MacFarlane <[email protected]>
Co-authored-by: Stefan Pingel <[email protected]>

* Migrate BFT tests to junit 5 (hyperledger#6350)

* bft tests to junit 5
* base class for pki extend AcceptanceTestBaseJunit5
* try/catch in case of empty optionals
* fixed parameterization method

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: Sally MacFarlane <[email protected]>

* fixing on selfdestruct (hyperledger#6359)

Signed-off-by: Karim Taam <[email protected]>

* migrate clique tests fully to junit5 (hyperledger#6362)

* migrate clique tests fully to junit5

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: Sally MacFarlane <[email protected]>

* fixed link to logging docs (hyperledger#6366)

Signed-off-by: Sally MacFarlane <[email protected]>

* Move logging to RunnerBuilder (hyperledger#6367)

Signed-off-by: Gabriel-Trintinalia <[email protected]>

* Use synchronized call to access the chain head block in `eth_estimateGas` (hyperledger#6345)

* Use synchronized call to access the chain head block in estimateGas()

Signed-off-by: Matthew Whitehead <[email protected]>

* Add error log entries when throwing internal error from estimateGas()

Signed-off-by: Matthew Whitehead <[email protected]>

* Update unit tests

Signed-off-by: Matthew Whitehead <[email protected]>

* Update changelog

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>

* Add --X-trie-log subcommand (hyperledger#6303)

* Add x-trie-log subcommand for one-off trie log backlog prune

Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Gabriel Fukushima <[email protected]>

---------

Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Gabriel Fukushima <[email protected]>
Co-authored-by: Simon Dudley <[email protected]>

* fix typos (hyperledger#6368)

Signed-off-by: vuittont60 <[email protected]>

* Added alias --sync-min-peers for --fast-sync-min-peers (hyperledger#6372)

* sync-min-peers as an alias
* added unit tests

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: Sally MacFarlane <[email protected]>

* Fix: Fallback to getName when canonicalName is null in BlockHeaderValidator DEBUG log (hyperledger#6332)

* fallback to simple name when canonical name is null
* use getName instead of getSimpleName to include the package name

Signed-off-by: Manoj P R <[email protected]>

---------

Signed-off-by: Manoj P R <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>

* fix: use UID 1000 for besu user (hyperledger#6358) (hyperledger#6360)

The openjdk-latest Docker image is using UID 1001 for besu, because its
base image ubuntu:23.10 now contains a default "ubuntu" user with UID
1000. (This UID change causes the besu user with UID 1001 to not have
access to files created for past versions with UID 1000.)

We now remove the default ubuntu user and explicitly use UID 1000 when
creating the besu user.

Signed-off-by: Hal Blackburn <[email protected]>

* Ignore generated files when running the spdx license check (hyperledger#6379)

Signed-off-by: Meredith Baxter <[email protected]>

* full sync - don't fail startup if sync-min-peers specified (hyperledger#6373)

Signed-off-by: Sally MacFarlane <[email protected]>

* Copy also computed fields, when doing a Transaction detached copy (hyperledger#6329)

Signed-off-by: Fabio Di Fabio <[email protected]>

* Disable txpool when not in sync (hyperledger#6302)


Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump to nex release snapshot 24.1.1 (hyperledger#6383)

* release next snapshot 24.1.1

Signed-off-by: garyschulte <[email protected]>

* move recent changelog items to 24.1.1-SNAPSHOT

Signed-off-by: garyschulte <[email protected]>

---------

Signed-off-by: garyschulte <[email protected]>

* Correct Tangerine Whistle definition in Fluent EVM APIs. (hyperledger#6382)

The fluent API incorrectly added the code size limit in Tangerine
Whistle instead of first adding it in Spurious Dragon.

Signed-off-by: Danno Ferrin <[email protected]>

* [MINOR] Fix pki tests condition check on mac (hyperledger#6387)

Signed-off-by: Gabriel-Trintinalia <[email protected]>

* Upgrade dependencies (hyperledger#6377)

* Bump com.github.oshi:oshi-core to 6.4.10

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump com.github.tomakehurst to org.wiremock 3.3.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump com.google.auto.service:auto-service to 1.1.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump com.google.dagger group to 2.50

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump com.graphql-java:graphql-java to 21.3

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump com.splunk.logging:splunk-library-javalogging to 1.11.8

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump com.squareup.okhttp3:okhttp to 4.12.0
Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump commons-io:commons-io to 2.15.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump dnsjava:dnsjava to 3.5.3

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump info.picocli group to 4.7.5

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump io.grpc group to 1.60.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump io.kubernetes:client-java to 18.0.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump io.netty group to 4.1.104.Final

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump net.java.dev.jna:jna to 5.14.0

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.apache.commons:commons-compress to 1.25.0

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.apache.commons:commons-lang3 to 3.14.0

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.apache.commons:commons-text to 1.11.0

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.apache.logging.log4j group to 2.22.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Redorder io.tmio group

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.assertj:assertj-core to 3.25.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.bouncycastle group to 1.77

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.fusesource.jansi:jansi to 2.4.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.immutables group 2.10.0

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.java-websocket:Java-WebSocket to 1.5.5

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.jetbrains.kotlin:kotlin-stdlib to 1.9.22

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.junit.jupiter group to 5.10.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.jupnp group to 2.7.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.rocksdb:rocksdbjni to 8.9.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.slf4j group to 2.0.10

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.springframework.security:spring-security-crypto to 6.2.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.testcontainers:testcontainers to 1.19.3

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.web3j group to 4.10.3

Signed-off-by: Fabio Di Fabio <[email protected]>

* Bump org.xerial.snappy:snappy-java to 1.1.10.5

Signed-off-by: Fabio Di Fabio <[email protected]>

* Regenerate gradle verification metadata

Signed-off-by: Fabio Di Fabio <[email protected]>

* Update commons-codec:commons-codec to 1.16.0

Signed-off-by: Fabio Di Fabio <[email protected]>

* Update org.junit.vintage:junit-vintage-engine to 5.10.1

Signed-off-by: Fabio Di Fabio <[email protected]>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <[email protected]>

---------

Signed-off-by: Fabio Di Fabio <[email protected]>

* add a fallback for docker detection on Mac (hyperledger#6356)

Signed-off-by: garyschulte <[email protected]>

* Fix test flackyness of acceptanceTestsPermissioning  (hyperledger#6384)


Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Sally MacFarlane <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>

* Upgrade `com.fasterxml.jackson` dependencies (hyperledger#6378)


Signed-off-by: Fabio Di Fabio <[email protected]>

* Use mining beneficiary from protocol spec in TraceServiceImpl (hyperledger#6390)

* use mining beneficiary from protocol spec

Signed-off-by: Daniel Lehrner <[email protected]>

---------

Signed-off-by: Daniel Lehrner <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>

* Update verification metadata and allowed licenses for Linea-Besu

---------

Signed-off-by: Karim Taam <[email protected]>
Signed-off-by: Sally MacFarlane <[email protected]>
Signed-off-by: jflo <[email protected]>
Signed-off-by: garyschulte <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Matthew Whitehead <[email protected]>
Signed-off-by: Gabriel-Trintinalia <[email protected]>
Signed-off-by: Simon Dudley <[email protected]>
Signed-off-by: Gabriel Fukushima <[email protected]>
Signed-off-by: vuittont60 <[email protected]>
Signed-off-by: Manoj P R <[email protected]>
Signed-off-by: Hal Blackburn <[email protected]>
Signed-off-by: Meredith Baxter <[email protected]>
Signed-off-by: Danno Ferrin <[email protected]>
Signed-off-by: Daniel Lehrner <[email protected]>
Co-authored-by: Karim TAAM <[email protected]>
Co-authored-by: garyschulte <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
Co-authored-by: Justin Florentine <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
Co-authored-by: Matt Whitehead <[email protected]>
Co-authored-by: Stefan Pingel <[email protected]>
Co-authored-by: Gabriel-Trintinalia <[email protected]>
Co-authored-by: Gabriel Fukushima <[email protected]>
Co-authored-by: Simon Dudley <[email protected]>
Co-authored-by: vuittont60 <[email protected]>
Co-authored-by: Manoj P R <[email protected]>
Co-authored-by: Hal Blackburn <[email protected]>
Co-authored-by: mbaxter <[email protected]>
Co-authored-by: Danno Ferrin <[email protected]>
Co-authored-by: daniellehrner <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Disable TransactionPool observer on new blocks during Full sync
3 participants