Skip to content

op-node: Simplify Receipts Fetching API#3639

Merged
trianglesphere merged 3 commits intodevelopfrom
jg/simplify_receipts
Oct 5, 2022
Merged

op-node: Simplify Receipts Fetching API#3639
trianglesphere merged 3 commits intodevelopfrom
jg/simplify_receipts

Conversation

@trianglesphere
Copy link
Contributor

Description

The previous api would return a receipts fetcher that the caller was responsible for iterating through and then calling Result on. This moves this inside the Fetch function. This simplifies usage of the API and does not result in performance regression because the receipts hash check requires all receipts to be fetched before

Tests

No added tests, but all pass.

@changeset-bot
Copy link

changeset-bot bot commented Oct 3, 2022

⚠️ No Changeset found

Latest commit: c54de7b

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@mergify
Copy link
Contributor

mergify bot commented Oct 3, 2022

This PR changes implementation code, but doesn't include a changeset. Did you forget to add one?

@tynes
Copy link
Contributor

tynes commented Oct 3, 2022

Generally seems like a sane change that makes using the L1ReceiptsFetcher easier, can't really think of a good reason to defer fetching the receipts like how the old code was architected

Copy link
Contributor

@protolambda protolambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want this? Sure it simplifies the API (yay, good thinking), but I expect it makes things quite unreliable when syncing from a L1 source: receipts are not cached individually, and the progress of the fetching is lost, since the receipt fetcher itself isn't cached anymore either.

Note that there's 1 receipt request per L1 tx. And on average Mainnet L1 has 10x as many txs per block as Goerli. And our op-e2e tx throughput is both lower and unrealistically stable (pretty much never a failed receipt retrieval).

Maybe we should do a mainnet test: without deploying any contracts, we can create a ghost rollup, that essentially functions the same like a rollup without active sequencer. Verifiers will run through all L1 blocks and receipts, and hit the bottlenecks that I think are fragile enough to warrant per-request caching/continuation on errors. If that works fine we can merge this, but I highly doubt it (especially when syncing from a rate-limited L1 endpoint, like an average user might...)

@trianglesphere
Copy link
Contributor Author

Do we really want this?

It's not an easy API to use and doesn't document that it relies on future calls to (receiptsFetcher).Fetch to work after a previous one didn't because Fetch hits a catch

receipts are not cached individually

That's not that hard to do.

@protolambda protolambda changed the title op-node: Simplify Reciepts Fetching API op-node: Simplify Receipts Fetching API Oct 4, 2022
@trianglesphere trianglesphere force-pushed the jg/simplify_receipts branch 2 times, most recently from 1e1eaf5 to 953a22b Compare October 4, 2022 17:54
@trianglesphere
Copy link
Contributor Author

@protolambda thanks for pointing out that the receipts fetcher cache is important. I've added a commit to this PR which now caches the receipts fetcher instead of the set of receipts. It attempts to used a cached receipts fetcher instead of making a new one. This means that this new code is the essentially the same as the old code except for the fact that the iteration is done inside of the Fetch call rather than outside.

How do you feel about this change? I think the performance should be the same and the API is a bit easier to use.

Copy link
Contributor

@protolambda protolambda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Hopefully the debug_getRawReceipts RPC method becomes more accessible than debug namespace some day, then the receipts fetcher stuff would be easier for us both. (And might still even be worth adding as option for those that sync from their own node now that I think of it)

@mergify
Copy link
Contributor

mergify bot commented Oct 5, 2022

This PR has been added to the merge queue, and will be merged soon.

@mergify
Copy link
Contributor

mergify bot commented Oct 5, 2022

This PR is next in line to be merged, and will be merged as soon as checks pass.

@mergify
Copy link
Contributor

mergify bot commented Oct 5, 2022

Merge failed. Please see automated check logs for more details.

@trianglesphere
Copy link
Contributor Author

gonna merge by hand b/c mergify messed up the update merge.

@mergify
Copy link
Contributor

mergify bot commented Oct 5, 2022

Hey @trianglesphere! This PR has merge conflicts. Please fix them before continuing review.

@mergify mergify bot added the conflict label Oct 5, 2022
@mergify mergify bot removed the conflict label Oct 5, 2022
@trianglesphere
Copy link
Contributor Author

Needed a force update b/c it conflicted with #3647 (b/c there are several interfaces that say the exact same thing).

The previous API would return a receipts fetcher that the caller
was responsible for iterating through and then calling `Result` on.
This moves this inside the `Fetch` function. This simplifies usage
of the API and does not result in performance regression because
the receipts hash check requires all receipts to be fetched before
This modifies the refactor to be equivalent to the previous code.
Caching the receipts fetcher is actually critical for performance
because the fetcher caches intermediate results. The fetcher typically
makes lots of calls to fetch all the receipts so if all work is thrown
out on a single failure it will be very hard for the node to make progress.
@trianglesphere trianglesphere merged commit c3afadb into develop Oct 5, 2022
@trianglesphere trianglesphere deleted the jg/simplify_receipts branch October 5, 2022 18:56
@trianglesphere trianglesphere restored the jg/simplify_receipts branch October 5, 2022 18:56
@trianglesphere trianglesphere deleted the jg/simplify_receipts branch October 5, 2022 18:56
This was referenced Oct 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants