Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(infra): Implement intelligent snapshots natively #194

Open
1 task done
antiyro opened this issue Jul 3, 2024 · 1 comment
Open
1 task done

feat(infra): Implement intelligent snapshots natively #194

antiyro opened this issue Jul 3, 2024 · 1 comment
Labels
enhancement New feature or request infra Infrastructure related label

Comments

@antiyro
Copy link
Member

antiyro commented Jul 3, 2024

Is there an existing issue?

  • I have searched the existing issues

Motivation

Currently, in Deoxys, the only way to synchronize the state is through the feeder gateway starting from block 0. This process can take several days, with or without rate limits, which can be a significant blocker for some developers. Therefore, we want to provide official pre-synced state snapshots that can be easily integrated at the node's startup.

Request

The purpose of this issue is to implement an intelligent snapshot system to enable faster synchronization by importing official pre-synced states. We would like to have a command --snap <block_number> that fetches the snapshot at the closest available height to the specified block number, downloads it to the user's defined --base-path, and decompresses it correctly with the corresponding logs.

Solution

To achieve this, we propose the following steps:

  1. Develop a command --snap <block_number> that identifies the closest available snapshot to the specified block number.
  2. Ensure the command downloads the snapshot to the path defined by the user's --base-path.
  3. Implement the functionality to decompress the snapshot correctly and log the process appropriately.

Are you willing to help with this request?

Yes!

@antiyro antiyro added enhancement New feature or request infra Infrastructure related label labels Jul 3, 2024
@antiyro antiyro added this to Deoxys Jul 3, 2024
@antiyro
Copy link
Member Author

antiyro commented Jul 3, 2024

The snapshots will be hosted in an object storage on OVH. You can simulate that during the development of the issue.

@antiyro antiyro moved this to 🏗 In progress in Deoxys Jul 3, 2024
@antiyro antiyro moved this from 🏗 In progress to 🆕 New in Deoxys Jul 3, 2024
@antiyro antiyro added this to Madara Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request infra Infrastructure related label
Projects
Status: Ready
Development

No branches or pull requests

1 participant