Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

op-service/txmgr: multiple fixes / improvements #11614

Merged

Conversation

roberto-bayardo
Copy link
Collaborator

@roberto-bayardo roberto-bayardo commented Aug 27, 2024

Description

  • have transactions get resubmitted even if we fail to bump fees to prevent indefinite hanging when a tx gets dropped from the mempool (as experienced by both base-sepolia and op-sepolia on 8/27/2024).

  • do not bump fees in response to errors that do not require fee bumping like nonce-too-low.

  • force a critical error if we get nonce-too-low before successfully publishing a transaction so the nonce state will reset before trying to publish another transaction.

  • set default send timeout to 10 minutes instead of infinite

  • some control flow refactoring to (hopefully) make the code a bit more understandable.

Tests

  • fixed existing unit tests that were relying on fee bumping when fees shouldn't have been bumped, and added new test for the immediate-fail-on-nonce-too-low case.

  • add test to ensure tx gets resubmitted even if fees can't be bumped.

  • add tests to check that both the network and send timeouts are appropriately respected when sending a transaction

@roberto-bayardo roberto-bayardo changed the title op-service/txmgr: make txmgr resubmit a transaction when fee bumping fails op-service/txmgr: fixes to none-too-low handling and fee bumping Aug 28, 2024
@roberto-bayardo roberto-bayardo changed the title op-service/txmgr: fixes to none-too-low handling and fee bumping op-service/txmgr: fixes to nonce-too-low handling and fee bumping Aug 28, 2024
@roberto-bayardo roberto-bayardo force-pushed the resend-after-fee-bump branch 19 times, most recently from a1eee31 to 95293ed Compare August 28, 2024 05:49
Copy link
Member

@sebastianst sebastianst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this fix the behavior we observed, where the cancellation tx mode was active in the batcher and it tried to cancel a transaction for a nonce that had actually already been confirmed on L1? In such a situation, the cancellation should immediately be aborted.

op-service/txmgr/txmgr.go Show resolved Hide resolved
op-service/txmgr/txmgr.go Outdated Show resolved Hide resolved
op-service/txmgr/txmgr.go Show resolved Hide resolved
op-service/txmgr/txmgr.go Show resolved Hide resolved
@roberto-bayardo
Copy link
Collaborator Author

Does this fix the behavior we observed, where the cancellation tx mode was active in the batcher and it tried to cancel a transaction for a nonce that had actually already been confirmed on L1? In such a situation, the cancellation should immediately be aborted.

Yes it fixes both the cases related to this. Note that in send_state I added a check that if nonce-too-low is returned by no tx has been successfully published, it will trigger an immediate abort.

In the problems we saw though, the issue was that we did successfully publish a transaction with that nonce at first. However since another one with the same nonce was confirmed in parallel, it was later pushed from the mempool. In this case, we also need the re-trying, which will ultimately start returning "nonce too low" and once it hits the abort-count (3 errors) it will abort.

@roberto-bayardo roberto-bayardo changed the title op-service/txmgr: fixes to nonce-too-low handling and fee bumping op-service/txmgr: multiple fixes Aug 28, 2024
@roberto-bayardo roberto-bayardo changed the title op-service/txmgr: multiple fixes op-service/txmgr: multiple fixes / improvements Aug 28, 2024
@roberto-bayardo roberto-bayardo force-pushed the resend-after-fee-bump branch 4 times, most recently from 6e8f288 to ae50124 Compare August 31, 2024 18:22
@roberto-bayardo roberto-bayardo marked this pull request as ready for review August 31, 2024 18:23
@roberto-bayardo roberto-bayardo requested a review from a team as a code owner August 31, 2024 18:23
@roberto-bayardo roberto-bayardo marked this pull request as draft August 31, 2024 18:52
@roberto-bayardo roberto-bayardo marked this pull request as ready for review August 31, 2024 19:21
@roberto-bayardo
Copy link
Collaborator Author

Ready for review!

@roberto-bayardo roberto-bayardo force-pushed the resend-after-fee-bump branch 2 times, most recently from d5e4bd6 to 940d4af Compare September 2, 2024 01:31
… succeed otherwise)

- make txmgr resubmit a transaction when fee bumping fails in case it has been dropped from the mempool

- only bump fees when they really should be bumped

- set txmgr overall default send timeout of 10 minutes. It was infinite, which led to permanently
  stuck transaction in combination with the other bugs fixed in this PR.
op-service/txmgr/txmgr_test.go Outdated Show resolved Hide resolved
@sebastianst sebastianst added this pull request to the merge queue Sep 2, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 2, 2024
@sebastianst sebastianst added this pull request to the merge queue Sep 2, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 2, 2024
@sebastianst sebastianst added this pull request to the merge queue Sep 2, 2024
Merged via the queue into ethereum-optimism:develop with commit 9452aa6 Sep 2, 2024
56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants