Skip to content

Comments

feat: round result from getSlot RPC method#48

Merged
lpahlavi merged 17 commits intomainfrom
lpahlavi/XC-292-get-slot-rounding
Apr 1, 2025
Merged

feat: round result from getSlot RPC method#48
lpahlavi merged 17 commits intomainfrom
lpahlavi/XC-292-get-slot-rounding

Conversation

@lpahlavi
Copy link
Contributor

@lpahlavi lpahlavi commented Mar 27, 2025

(XC-292) Round the result from the Solana getSlot RPC method to the nearest n blocks where n is a user parameter given as an input to the canister method. This is done to partly circumvent the fast Solana block time and achieve a higher consensus rate when performing the HTTP outcalls to fetch the latest slot.

@lpahlavi lpahlavi marked this pull request as ready for review March 27, 2025 15:47
@lpahlavi lpahlavi requested a review from a team as a code owner March 27, 2025 15:47
#[n(0)]
GetSlot,
#[n(1)]
GetSlot(#[n(1)] u64),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's have a proper RoundingError struct that wraps a u64. Thanks to minicbor, this comes with little additional overhead in the bytes size. e.g,

[derive(Debug, Decode, Encode)]
pub enum ResponseTransform {
    #[n(0)]
    GetSlot(#[n(1)] RoundingError),
    ...
}
[derive(Debug, Decode, Encode)]
pub struct RoundingError(#[n(0)] u64)

Like this one can also implement Default on RoundingError and remove the constant DEFAULT_ROUNDING_ERROR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. As discussed, I think this doesn't need to be in the Candid interface (annoying to use for end users since they then have to wrap the value in a new type), however I think this makes sense to use for the client. For this reason I added it to the sol_rpc_types crate including some unit tests. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding RoundingError!

however I think this makes sense to use for the client. For this reason I added it to the sol_rpc_type

This part is less clear to me. Why not just have a u64 in the client? I find it a bit weird to have it in sol_rpc_types given that's it's not a Candid type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could definitely go both ways. My thought process though, is that whereas in the Candid interface we want to simplify things as much as possible to make e.g. CLI commands with dfx easy, with the client it is nice to use types to leverage Rust's typing system in our favor. Here, for example, there is the advantage of having documentation on the RoundingError type with examples etc. IMO this makes it more explicit and better documented what the rounding_error parameter actually does. However, it's true that it is a bit more different from the Candid interface, and maybe it's acceptable to expect developers look there for documentation? WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the current situation is a bit too inconsistent:

  1. The Candid interface takes a u64
  2. The client takes a dedicated struct RoundingError

I would have public-facing methods (directly calling the SOL RPC canister or using the client) either use u64 or RoundingError but not both. I currently would tend to use a u64:

  1. It's usually beneficial for public-facing APIs to be "generous" in the types of parameters they accept (but be restrictive in the return types). This the same argument as the convenience you mentioned for being able to easily use it with dfx.
  2. Minimize conversion: now the client takes a RoundingError but needs to convert it back into a u64 to fit it in GetSlotRpcConfig.
  3. For the doc, we could point to the doc on the Candid interface.

I'm also ok with both using RoundingError, just not the current mix (happy to discuss online).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good points. I guess you are right that it makes sense to try and keep a consistent interface between the Candid canister interface and the client interface. I think the Candid interface probably benefits more from the simple u64 type so I removed the RoundingError type from the client method signature and moved it to the canister crate. Let me know if you have any other comments/concerns.

Copy link
Contributor

@gregorydemay gregorydemay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lpahlavi for improving the PR! Mostly some minor comments and some understanding question

#[n(0)]
GetSlot,
#[n(1)]
GetSlot(#[n(1)] u64),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding RoundingError!

however I think this makes sense to use for the client. For this reason I added it to the sol_rpc_type

This part is less clear to me. Why not just have a u64 in the client? I find it a bit weird to have it in sol_rpc_types given that's it's not a Candid type.

Copy link
Contributor Author

@lpahlavi lpahlavi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the feedback @gregorydemay, LMK what you think of the new changes.

#[n(0)]
GetSlot,
#[n(1)]
GetSlot(#[n(1)] u64),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could definitely go both ways. My thought process though, is that whereas in the Candid interface we want to simplify things as much as possible to make e.g. CLI commands with dfx easy, with the client it is nice to use types to leverage Rust's typing system in our favor. Here, for example, there is the advantage of having documentation on the RoundingError type with examples etc. IMO this makes it more explicit and better documented what the rounding_error parameter actually does. However, it's true that it is a bit more different from the Candid interface, and maybe it's acceptable to expect developers look there for documentation? WDYT?

Copy link
Contributor

@gregorydemay gregorydemay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @lpahlavi for adding the tests. There is the opened question of where to put RoundingError but otherwise LGTM!

#[n(0)]
GetSlot,
#[n(1)]
GetSlot(#[n(1)] u64),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the current situation is a bit too inconsistent:

  1. The Candid interface takes a u64
  2. The client takes a dedicated struct RoundingError

I would have public-facing methods (directly calling the SOL RPC canister or using the client) either use u64 or RoundingError but not both. I currently would tend to use a u64:

  1. It's usually beneficial for public-facing APIs to be "generous" in the types of parameters they accept (but be restrictive in the return types). This the same argument as the convenience you mentioned for being able to easily use it with dfx.
  2. Minimize conversion: now the client takes a RoundingError but needs to convert it back into a u64 to fit it in GetSlotRpcConfig.
  3. For the doc, we could point to the doc on the Candid interface.

I'm also ok with both using RoundingError, just not the current mix (happy to discuss online).

lpahlavi and others added 2 commits April 1, 2025 13:31
Co-authored-by: gregorydemay <112856886+gregorydemay@users.noreply.github.com>
Copy link
Contributor Author

@lpahlavi lpahlavi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gregorydemay Thanks a lot for the multiple rounds of review and helpful feedback! Let me know if you have any further questions/comments/concerns.

#[n(0)]
GetSlot,
#[n(1)]
GetSlot(#[n(1)] u64),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good points. I guess you are right that it makes sense to try and keep a consistent interface between the Candid canister interface and the client interface. I think the Candid interface probably benefits more from the simple u64 type so I removed the RoundingError type from the client method signature and moved it to the canister crate. Let me know if you have any other comments/concerns.

Copy link
Contributor

@gregorydemay gregorydemay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the changes! Would you mind looking for any TODO XC-292 before merging? There should be at least one here

@lpahlavi lpahlavi merged commit d2bb7a8 into main Apr 1, 2025
7 checks passed
@lpahlavi lpahlavi deleted the lpahlavi/XC-292-get-slot-rounding branch April 1, 2025 13:05
@github-actions github-actions bot mentioned this pull request Apr 28, 2025
@github-actions github-actions bot mentioned this pull request Apr 29, 2025
gregorydemay added a commit that referenced this pull request Apr 29, 2025
## 🤖 New release

* `sol_rpc_types`: 0.1.0
* `sol_rpc_canister`: 0.1.0
* `sol_rpc_client`: 0.1.0

<details><summary><i><b>Changelog</b></i></summary><p>

## `sol_rpc_types`

<blockquote>

## [0.1.0] - 2025-04-29

### Added

- Add Rustdoc for types related to `getTransaction`
([#71](#71))
- Add support for `getTransaction` RPC method
([#68](#68))
- Add `getBlock` RPC method
([#53](#53))
- Add `sendTransaction` RPC method
([#59](#59))
- Add NOTICE to Apache license
([#60](#60))
- Add `getAccountInfo` RPC method
([#49](#49))
- Add metrics
([#41](#41))
- Add logging crate
([#13](#13))
- Add doc build stage to CI
([#19](#19))
- Add support for override providers
([#12](#12))

### Changed

- Use `canlog_derive` and `canlog` from crates.io
([#84](#84))
- Release pipeline
([#4](#4))
- Default commitment level for `SolRpcClient`
([#77](#77))
- Support method `getBalance`
([#74](#74))
- Rename some enum variants to camel case when serializing
([#72](#72))
- Use SOL RPC canister in `basic_solana` example
([#69](#69))
- Add basic README instructions
([#63](#63))
- Cycles cost
([#52](#52))
- Client builder
([#54](#54))
- Round result from `getSlot` RPC method
([#48](#48))
- Use `canhttp` `multi` feature
([#46](#46))
- Implement getSlot RPC method
([#33](#33))
- Add some tested RPC providers for Solana Mainnet and Devnet
([#15](#15))
- Streamline providers
([#32](#32))
- Add support for API keys
([#10](#10))
- Hard-code SOL RPC providers
([#9](#9))
- Basic Solana wallet example
([#1](#1))
- Initial cargo workspace and build pipeline
([#2](#2))

### Fixed

- Correct Solana cluster for dRPC and Helius providers
([#47](#47))


<!-- generated by git-cliff -->
Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a
Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic
Versioning](https://semver.org/spec/v2.0.0.html).
</blockquote>

## `sol_rpc_canister`

<blockquote>

## [0.1.0] - 2025-04-29

### Added

- Add support for `getTransaction` RPC method
([#68](#68))
- Add `getBlock` RPC method
([#53](#53))
- Add `sendTransaction` RPC method
([#59](#59))
- Add NOTICE to Apache license
([#60](#60))
- Add `getAccountInfo` RPC method
([#49](#49))
- Add metrics
([#41](#41))
- Add logging crate
([#13](#13))
- Add support for override providers
([#12](#12))

### Changed

- Use `canlog_derive` and `canlog` from crates.io
([#84](#84))
- Release pipeline
([#4](#4))
- Clean-up TODOs
([#81](#81))
- Support method `getBalance`
([#74](#74))
- Remove http_types module and use external ic-http-types crate
([#73](#73))
- Rename some enum variants to camel case when serializing
([#72](#72))
- Use constant size JSON-RPC request ID
([#62](#62))
- Use method from JSON-RPC request for metric
([#61](#61))
- Cycles cost
([#52](#52))
- Client builder
([#54](#54))
- Round result from `getSlot` RPC method
([#48](#48))
- Use `canhttp` `multi` feature
([#46](#46))
- Implement a method for making generic RPC request
([#39](#39))
- Implement getSlot RPC method
([#33](#33))
- Add some tested RPC providers for Solana Mainnet and Devnet
([#15](#15))
- Streamline providers
([#32](#32))
- Update rust toolchain to 1.85
([#21](#21))
- Remove unnecessary Storable implementations
([#14](#14))
- Add support for API keys
([#10](#10))
- Hard-code SOL RPC providers
([#9](#9))
- Reproducible build
([#3](#3))
- Initial cargo workspace and build pipeline
([#2](#2))

### Fixed

- Set `maxSupportedTransactionVersion` to zero for end-to-end tests
([#85](#85))
- API keys ([#58](#58))
- End-to-end tests
([#45](#45))
- Correct Solana cluster for dRPC and Helius providers
([#47](#47))
- E2e test with Solana test validator
([#20](#20))
- Create test canister on ICP mainnet
([#8](#8))


<!-- generated by git-cliff -->
Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a
Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic
Versioning](https://semver.org/spec/v2.0.0.html).
</blockquote>

## `sol_rpc_client`

<blockquote>

## [0.1.0] - 2025-04-29

### Added

- Add support for `getTransaction` RPC method
([#68](#68))
- Add `getBlock` RPC method
([#53](#53))
- Add `sendTransaction` RPC method
([#59](#59))
- Add NOTICE to Apache license
([#60](#60))
- Add `getAccountInfo` RPC method
([#49](#49))

### Changed

- Release pipeline
([#4](#4))
- Default commitment level for `SolRpcClient`
([#77](#77))
- Support method `getBalance`
([#74](#74))
- Use SOL RPC canister in `basic_solana` example
([#69](#69))
- Rust documentation tests for the `SolRpcClient`
([#65](#65))
- Add basic README instructions
([#63](#63))
- Cycles cost
([#52](#52))
- Client builder
([#54](#54))
- Round result from `getSlot` RPC method
([#48](#48))
- Use `canhttp` `multi` feature
([#46](#46))
- Implement a method for making generic RPC request
([#39](#39))
- Implement getSlot RPC method
([#33](#33))
- Streamline providers
([#32](#32))
- Add support for API keys
([#10](#10))
- Hard-code SOL RPC providers
([#9](#9))
- Initial cargo workspace and build pipeline
([#2](#2))

### Fixed

- Forward calls through wallet canister
([#40](#40))
- E2e test with Solana test validator
([#20](#20))


<!-- generated by git-cliff -->
Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a
Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic
Versioning](https://semver.org/spec/v2.0.0.html).
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

---------

Co-authored-by: gregorydemay <gregory.demay@dfinity.org>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
gregorydemay added a commit that referenced this pull request May 14, 2025
Add support for the method
[`getRecentPrioritizationFees`](https://solana.com/de/docs/rpc/http/getrecentprioritizationfees).

Supporting this method via HTTPs outcalls offers similar challenges to
be able to achieve consensus on the responses as for supporting the
method [`getSlot`](https://solana.com/de/docs/rpc/http/getslot) (#33,
#48). Specifically,

1. Each response to `getRecentPrioritizationFees` consists of a
fast-changing subset of slots, where each slot is associated with a
[prioritization
fee](https://solana.com/de/developers/guides/advanced/how-to-use-priority-fees).
Responses between nodes may vary because the response contains
`processed` slots and a new `processed` slot is produced roughly every
400ms.
2. Similarly to `getSlot`, we need a heuristic to select
deterministically a subset of slots in the response seen by a node that
at least two thirds of nodes can agree on with a significant
probability, which proceeds as follows:
    1. Sort the prioritization fees according to their slot.
2. Agree on the last processed slot. This uses rounding, exactly as for
`getSlot`.
3. Agree on the size of the subset that will contain the prioritization
fee of the previously-agreed last processed slot and the remaining fees
by decreasing order of slot (higher is more recent).
@github-actions github-actions bot mentioned this pull request May 27, 2025
gregorydemay added a commit that referenced this pull request May 27, 2025
## 🤖 New release

* `sol_rpc_canister`: 0.2.0

<details><summary><i><b>Changelog</b></i></summary><p>

<blockquote>

## [0.2.0] - 2025-05-27

### Added

- Add client method to sign a transaction
([#113](#113))
- Add Chainstack RPC provider
([#118](#118))
- Add support for `getSignaturesForAddress`
([#106](#106))
- Add support for `getSignatureStatuses` RPC method
([#96](#96))
- Add support for `getTokenAccountBalance` RPC method
([#90](#90))
- Add support for `getTransaction` RPC method
([#68](#68))
- Add `getBlock` RPC method
([#53](#53))
- Add `sendTransaction` RPC method
([#59](#59))
- Add NOTICE to Apache license
([#60](#60))
- Add `getAccountInfo` RPC method
([#49](#49))
- Add metrics
([#41](#41))
- Add logging crate
([#13](#13))
- Add support for override providers
([#12](#12))

### Changed

- Bump version to 0.2
- Candid NonZeroU8
([#108](#108))
- Add `RoundingError` to `sol_rpc_types`
([#105](#105))
- Use secure primitive types for `Pubkey`, `Signature` and `Hash`
([#98](#98))
- Add support for `getRecentPrioritizationFees`
([#92](#92))
- Simplify API keys provisioning script
([#89](#89))
- Release v0.1.0
([#88](#88))
- Use `canlog_derive` and `canlog` from crates.io
([#84](#84))
- Release pipeline
([#4](#4))
- Clean-up TODOs
([#81](#81))
- Support method `getBalance`
([#74](#74))
- Remove http_types module and use external ic-http-types crate
([#73](#73))
- Rename some enum variants to camel case when serializing
([#72](#72))
- Use constant size JSON-RPC request ID
([#62](#62))
- Use method from JSON-RPC request for metric
([#61](#61))
- Cycles cost
([#52](#52))
- Client builder
([#54](#54))
- Round result from `getSlot` RPC method
([#48](#48))
- Use `canhttp` `multi` feature
([#46](#46))
- Implement a method for making generic RPC request
([#39](#39))
- Implement getSlot RPC method
([#33](#33))
- Add some tested RPC providers for Solana Mainnet and Devnet
([#15](#15))
- Streamline providers
([#32](#32))
- Update rust toolchain to 1.85
([#21](#21))
- Remove unnecessary Storable implementations
([#14](#14))
- Add support for API keys
([#10](#10))
- Hard-code SOL RPC providers
([#9](#9))
- Reproducible build
([#3](#3))
- Initial cargo workspace and build pipeline
([#2](#2))

### Fixed

- Missing `TraceHttp` logs
([#129](#129))
- End-to-end tests for `sendTransaction`
([#104](#104))
- Unit test for `getRecentPrioritizationFees` parameters serialization
([#107](#107))
- Integration test for `verifyApiKey`
([#82](#82))
- Set `maxSupportedTransactionVersion` to zero for end-to-end tests
([#85](#85))
- API keys ([#58](#58))
- End-to-end tests
([#45](#45))
- Correct Solana cluster for dRPC and Helius providers
([#47](#47))
- E2e test with Solana test validator
([#20](#20))
- Create test canister on ICP mainnet
([#8](#8))

### Removed

- Remove default/non-default providers
([#122](#122))
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

---------

Co-authored-by: gregorydemay <gregory.demay@dfinity.org>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
@github-actions github-actions bot mentioned this pull request Jun 12, 2025
gregorydemay added a commit that referenced this pull request Jun 12, 2025
## 🤖 New release

* `sol_rpc_canister`: 1.0.0

<details><summary><i><b>Changelog</b></i></summary><p>

<blockquote>

## [1.0.0] - 2025-06-12

### Added

- Add support for `transactionDetails=accounts`
([#139](#139))
- Add support for `rewards` parameter for `getBlock`
([#135](#135))
- Add helper methods for request builders
([#136](#136))
- Add client method to sign a transaction
([#113](#113))
- Add Chainstack RPC provider
([#118](#118))
- Add support for `getSignaturesForAddress`
([#106](#106))
- Add support for `getSignatureStatuses` RPC method
([#96](#96))
- Add support for `getTokenAccountBalance` RPC method
([#90](#90))
- Add support for `getTransaction` RPC method
([#68](#68))
- Add `getBlock` RPC method
([#53](#53))
- Add `sendTransaction` RPC method
([#59](#59))
- Add NOTICE to Apache license
([#60](#60))
- Add `getAccountInfo` RPC method
([#49](#49))
- Add metrics
([#41](#41))
- Add logging crate
([#13](#13))
- Add support for override providers
([#12](#12))

### Changed

- Bump to 1.0.0
- Inline request parameters `is_default_config` methods
([#138](#138))
- Release v0.2.0
([#131](#131))
- Bump version and use a release notes template
([#130](#130))
- Candid NonZeroU8
([#108](#108))
- Add `RoundingError` to `sol_rpc_types`
([#105](#105))
- Use secure primitive types for `Pubkey`, `Signature` and `Hash`
([#98](#98))
- Add support for `getRecentPrioritizationFees`
([#92](#92))
- Simplify API keys provisioning script
([#89](#89))
- Release v0.1.0
([#88](#88))
- Use `canlog_derive` and `canlog` from crates.io
([#84](#84))
- Release pipeline
([#4](#4))
- Clean-up TODOs
([#81](#81))
- Support method `getBalance`
([#74](#74))
- Remove http_types module and use external ic-http-types crate
([#73](#73))
- Rename some enum variants to camel case when serializing
([#72](#72))
- Use constant size JSON-RPC request ID
([#62](#62))
- Use method from JSON-RPC request for metric
([#61](#61))
- Cycles cost
([#52](#52))
- Client builder
([#54](#54))
- Round result from `getSlot` RPC method
([#48](#48))
- Use `canhttp` `multi` feature
([#46](#46))
- Implement a method for making generic RPC request
([#39](#39))
- Implement getSlot RPC method
([#33](#33))
- Add some tested RPC providers for Solana Mainnet and Devnet
([#15](#15))
- Streamline providers
([#32](#32))
- Update rust toolchain to 1.85
([#21](#21))
- Remove unnecessary Storable implementations
([#14](#14))
- Add support for API keys
([#10](#10))
- Hard-code SOL RPC providers
([#9](#9))
- Reproducible build
([#3](#3))
- Initial cargo workspace and build pipeline
([#2](#2))

### Fixed

- Missing `TraceHttp` logs
([#129](#129))
- End-to-end tests for `sendTransaction`
([#104](#104))
- Unit test for `getRecentPrioritizationFees` parameters serialization
([#107](#107))
- Integration test for `verifyApiKey`
([#82](#82))
- Set `maxSupportedTransactionVersion` to zero for end-to-end tests
([#85](#85))
- API keys ([#58](#58))
- End-to-end tests
([#45](#45))
- Correct Solana cluster for dRPC and Helius providers
([#47](#47))
- E2e test with Solana test validator
([#20](#20))
- Create test canister on ICP mainnet
([#8](#8))

### Removed

- Remove default/non-default providers
([#122](#122))
</blockquote>


</p></details>

---
This PR was generated with
[release-plz](https://github.com/release-plz/release-plz/).

---------

Co-authored-by: gregorydemay <gregory.demay@dfinity.org>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: gregorydemay <112856886+gregorydemay@users.noreply.github.com>
@github-actions github-actions bot mentioned this pull request Jul 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants