feat: Store limited number of expiring elements#434
Merged
gregorydemay merged 24 commits intomainfrom Jul 1, 2025
Merged
Conversation
gregorydemay
force-pushed
the
gdemay/XC-372-select-providers
branch
from
0a25879 to
851d620
Compare
gregorydemay
changed the base branch from
main
to
gdemay/XC-372-supported-provider
gregorydemay
changed the title
lpahlavi
reviewed
Jun 20, 2025
Contributor
lpahlavi
left a comment
lpahlavi
left a comment
There was a problem hiding this comment.
Thanks for the PR @gregorydemay! Generally looks very cool! Mostly some small comments/suggestions from my side.
gregorydemay
added a commit
that referenced
this pull request
Jun 23, 2025
The notion of default/non-default providers was only useful when Equality was chosen as consensus strategy and no providers was explicitly specified, in which case the "default" providers would be selected. This PR removes the notion of default/non-default providers by replacing with an ordered list of providers for a given EVM chain, corresponding to the supported providers. The previous notion of "default" providers simply correspond to the first three providers in that list. This is similar to the refactoring done in dfinity/sol-rpc-canister#122. Differentiating between supported and custom providers is the first step towards dynamically ranking supported providers according to the number of their successful responses, which is done in the following 3 PRs: 1. #436 (this one) 2. #434 3. #435
# Conflicts: # src/rpc_client/mod.rs
gregorydemay
commented
Jun 23, 2025
Contributor
Author
gregorydemay
left a comment
gregorydemay
left a comment
There was a problem hiding this comment.
Thanks @lpahlavi for the review and the suggestions!
Contributor
|
Thanks for the insightful explanations and changes @gregorydemay! There is still one minor comment but otherwise looks good! |
gregorydemay
commented
Jun 24, 2025
lpahlavi
approved these changes
Jun 24, 2025
Contributor
lpahlavi
left a comment
lpahlavi
left a comment
There was a problem hiding this comment.
Thanks a lot for adding the doc @gregorydemay! Some small typos in the docs but otherwise looks great!
ninegua
reviewed
Jun 24, 2025
ninegua
reviewed
Jun 24, 2025
Contributor
Author
|
CI is currently blocked because of BlockPi, should be unblocked with #440. |
gregorydemay
added a commit
that referenced
this pull request
Jul 4, 2025
Dynamically select supported JSON-RPC providers according to the number of successful JSON-RPC responses in the last 20 minutes, where providers with a higher number will be prioritized over ones with lower numbers. This is a heuristic that aims at selecting only currently functioning providers. The idea is that since JSON-RPC errors cannot be relied upon, as they are not specified by the Ethereum JSON-RPC API and so different EVM clients or JSON-RPC providers do use different errors, we only count instead the number of successful responses. Obviously, this is still not bulletproof, since it won't help in case the provider returns a successful JSON-RPC response with incorrect data. Looking back at previous incidents with Ethereum JSON-RPC providers for the EVM RPC canister or the ckETH/ckERC20 minter, we can see that having this heuristic would have helped in most cases, excepted when a provider returns successful responses with incorrect data (1 out of 4 incidents): 1. 😌 `Cloudflare` always return an error (proposal [136701](https://dashboard.internetcomputer.org/proposal/136701)) 2. 😌 `LlamaNodes` is down (proposal [132474](https://dashboard.internetcomputer.org/proposal/132474)) 3. 😌 `Ankr` dropped IPv6 connectivity ([132415](https://dashboard.internetcomputer.org/proposal/132415) 4. 😱`Cloudflare` returns successful responses with wrong results (proposal [128365](https://dashboard.internetcomputer.org/proposal/128365)) This is the last step towards dynamically ranking supported providers according to the number of their successful responses, which is done in the following 3 PRs: 1. #436 5. #434 6. #435 (this one)
gregorydemay
added a commit
to dfinity/canhttp
that referenced
this pull request
Jul 9, 2025
…ster#434) Introduce a new data structure `TimedSizedVec<T>` to store a limited number of values, evicting older and expired entries. This is similar to [`TimedSizedCache`](https://docs.rs/cached/0.55.1/cached/stores/struct.TimedSizedCache.html) except that it's time agnostic to ensure that it works in a canister (no `Instant::now`). In a later PR, this data structure will be used to register timestamps of JSON-RPC ok responses from each supported provider. This is the second step towards dynamically ranking supported providers according to the number of their successful responses, which is done in the following 3 PRs: 1. dfinity/evm-rpc-canister#436 2. dfinity/evm-rpc-canister#434 (this one) 3. dfinity/evm-rpc-canister#435 --------- Co-authored-by: Louis Pahlavi <louis.pahlavi@gmail.com>
gregorydemay
added a commit
to dfinity/canhttp
that referenced
this pull request
Jul 9, 2025
…ty/evm-rpc-canister#435) Dynamically select supported JSON-RPC providers according to the number of successful JSON-RPC responses in the last 20 minutes, where providers with a higher number will be prioritized over ones with lower numbers. This is a heuristic that aims at selecting only currently functioning providers. The idea is that since JSON-RPC errors cannot be relied upon, as they are not specified by the Ethereum JSON-RPC API and so different EVM clients or JSON-RPC providers do use different errors, we only count instead the number of successful responses. Obviously, this is still not bulletproof, since it won't help in case the provider returns a successful JSON-RPC response with incorrect data. Looking back at previous incidents with Ethereum JSON-RPC providers for the EVM RPC canister or the ckETH/ckERC20 minter, we can see that having this heuristic would have helped in most cases, excepted when a provider returns successful responses with incorrect data (1 out of 4 incidents): 1. 😌 `Cloudflare` always return an error (proposal [136701](https://dashboard.internetcomputer.org/proposal/136701)) 2. 😌 `LlamaNodes` is down (proposal [132474](https://dashboard.internetcomputer.org/proposal/132474)) 3. 😌 `Ankr` dropped IPv6 connectivity ([132415](https://dashboard.internetcomputer.org/proposal/132415) 4. 😱`Cloudflare` returns successful responses with wrong results (proposal [128365](https://dashboard.internetcomputer.org/proposal/128365)) This is the last step towards dynamically ranking supported providers according to the number of their successful responses, which is done in the following 3 PRs: 1. dfinity/evm-rpc-canister#436 5. dfinity/evm-rpc-canister#434 6. dfinity/evm-rpc-canister#435 (this one)
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a new data structure
TimedSizedVec<T>to store a limited number of values, evicting older and expired entries. This is similar toTimedSizedCacheexcept that it's time agnostic to ensure that it works in a canister (noInstant::now).In a later PR, this data structure will be used to register timestamps of JSON-RPC ok responses from each supported provider. This is the second step towards dynamically ranking supported providers according to the number of their successful responses, which is done in the following 3 PRs: