Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Listener catch-up fix #450

Merged
merged 6 commits into from
Aug 21, 2024
Merged

Listener catch-up fix #450

merged 6 commits into from
Aug 21, 2024

Conversation

iansuvak
Copy link
Contributor

@iansuvak iansuvak commented Aug 21, 2024

Why this should be merged

Addresses #443

How this works

  • Switches catchup from using WS client that is available on the subscriber level to the RPC client used on the listener level. This means that WS issues that subscriber handles by reconnecting should no longer affect the catchup mechanism
  • Moves catchup process from subscriber to listener along with the headers channel
  • Adds simple retries to the network request.

After doing this I realized that this change can be made lighter by passing in an RPC client to the Subscriber as well. That would mean that we can keep the catchup and the headers on the subscriber level.
I pivoted to keeping things on the subscriber level and just changing the client and adding retries there

How this was tested

CI

How is this documented

N/A

relayer/catchup_listener.go Outdated Show resolved Hide resolved
vms/evm/subscriber.go Outdated Show resolved Hide resolved
Copy link
Contributor

@bernard-avalabs bernard-avalabs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good to me. Just left a minor comment about timeouts.

)
// Sleep if this wasn't the last retry
if i < rpcMaxRetries-1 {
time.Sleep(time.Duration(i+1) * time.Second)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it might be more clear to specify the total timeout/sleep duration (10 seconds across 5 retries) as a constant. Any reason why the subscriber timeout is a constant on each retry, but the rpc timeout grows linearly with each retry?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, we should be consistent here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree but in which direction? constant? Ideally with network resources I like exponential backoffs with jitter but seemed like an overkill here.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exponential backoff has come up in previous discussions, but we've come to similar conclusions that it's overkill. Since the overall number of RPC calls is low, optimizing how they're spaced hasn't been a top priority.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we were tracking that already, but apparently not. I created #453 to do so.

vms/evm/subscriber.go Outdated Show resolved Hide resolved
var err error
var header *types.Header
attempt := 1
for {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't we do a regular for attempt=1; attempt < retryAttempts; attempt ++ here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matching the way that subscriber reattempts right below this

Co-authored-by: cam-schultz <[email protected]>
Signed-off-by: Ian Suvak <[email protected]>
@iansuvak iansuvak merged commit d104e65 into main Aug 21, 2024
7 checks passed
@iansuvak iansuvak deleted the move-catchup-to-listener branch August 21, 2024 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants