feat(indexer) consolidate retrieval of finalized headers

[hamdiallam]

The Fetcher will be used to drive new batches of headers to check. This can be used against both the L1 and L2 instances. Not super clear why the indexer currently duplicates this logic in both L1/L2

Follow up

1. Implement the processor which will poll the Fetcher
2. Investigate if we can utilize a subscription for new headers instead of polling
3. More robust implementation for the EthClient interface. See some of the comments on this PR

[changeset-bot]

⚠️ No Changeset found

Latest commit: 97de2f2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[netlify]

✅ Deploy Preview for opstack-docs canceled.

Name	Link
🔨 Latest commit	97de2f2
🔍 Latest deploy log	https://app.netlify.com/sites/opstack-docs/deploys/647e25a4bce82d0008755176

[tynes]

Could reuse this tooling for the batch calling:

optimism/op-node/client/rpc.go

Line 144 in 2d04a15

func (b *BaseRPCClient) BatchCallContext(ctx context.Context, batch []rpc.BatchElem) error {

I think this is what you'd want to use:

optimism/op-node/client/rpc.go

Line 162 in 2d04a15

func NewInstrumentedRPC(c RPC, m *metrics.Metrics) *InstrumentedRPCClient {

[hamdiallam]

Could reuse this tooling for the batch calling:

optimism/op-node/client/rpc.go

Line 144 in 2d04a15

func (b *BaseRPCClient) BatchCallContext(ctx context.Context, batch []rpc.BatchElem) error {

I think this is what you'd want to use:

optimism/op-node/client/rpc.go

Line 162 in 2d04a15

func NewInstrumentedRPC(c RPC, m *metrics.Metrics) *InstrumentedRPCClient {

ooh nice this is helpful thanks! I'll revisit a lot of the EthClient implementation when we incorporating retries/backoff

[roninjin10]

https://linear.app/optimism/issue/DX-50/indexer-create-event-ingestor

[roninjin10]

DX-50

[roninjin10]

FE-1197 does it work for ecosystem team?

[hamdiallam]

Updated so that the fetcher holds reference to the last header. Useful so that the fetcher can ensure the next batch is chained correctly on the last header

[hamdiallam]

Copying @roninjin10 comment about batch sizes

Just sharing context on how others do this:

Ponder fetches an extremely aggressive amount at once and then if it fails backs off to more conservative amounts.

The graph does the opposite where it tries to get more aggressive as it goes

[mergify]

This PR has been added to the merge queue, and will be merged soon.

[mergify]

This PR is next in line to be merged, and will be merged as soon as checks pass.

[mergify]

This PR is next in line to be merged, and will be merged as soon as checks pass.