Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rpc-v2: Implement archive_unstable_storageDiff #5997

Merged
merged 60 commits into from
Nov 27, 2024
Merged

rpc-v2: Implement archive_unstable_storageDiff #5997

merged 60 commits into from
Nov 27, 2024

Conversation

lexnv
Copy link
Contributor

@lexnv lexnv commented Oct 9, 2024 •
edited
Loading

This PR implements the archive_unstable_storageDiff.

The implementation follows the rpc-v2 spec from:

cc @paritytech/subxt-team

lexnv added 8 commits October 9, 2024 10:54
@lexnv
archive/api: Add stroage diff method
e17a555 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
storage/events: Add storage events and test serialization
4101ee6 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Add initial implementation for storage diff
d781d67 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Handle modified and delete cases
60e1a93 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/events: Derive more impl for diff items
63da031 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Deduplicate input keys
25b9e5a 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Move code to different module
52cfaf7 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Fetch value / hash or both
ee6bed7 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv lexnv added I5-enhancement An additional feature request. T3-RPC_API This PR/Issue is related to RPC APIs. D1-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase. labels Oct 9, 2024
@lexnv lexnv self-assigned this Oct 9, 2024
lexnv added 18 commits November 4, 2024 18:40
@lexnv
events: Add subscription events for ArchiveStorageDiffEvent
8948e74 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Make archive_storageDiff a subscription
fbfb0dc 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Modify storageDiff to leverage subscription backpressure
f7b33ed 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Deduplicate items to a separate function
f47db36 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/tests: Check deduplication
97546a4 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Fix deduplication shortest key
baa3dd0 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/tests: Check complex deduplication
4683f6b 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Simplify logic of trie iteration
36f59d2 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Improve documentation
a084f47 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/events: Add derive(Eq) to archive storage diff events
77cac50 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/tests: Check query for main trie under prefix
e2324a4 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Add trace and debug logs for storage diff
4b05913 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Send StorageDiffDone for partial trie queries
83cb9b4 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/tests: Check no changes between blocks
85c810e 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/tests: Ensure added key
152d13b 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/tests: Extend test with interleaved values and hashes
4f549bf 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
rpc-v2: Use indexmap crate
d208068 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Move prevHash as last parameter to archive diff
369a4d2 
Signed-off-by: Alexandru Vasile <[email protected]>
jsdw
jsdw approved these changes Nov 21, 2024
View reviewed changes
Copy link
Contributor

@jsdw jsdw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments but overall looks great!

lexnv and others added 10 commits November 26, 2024 13:23
@lexnv
Merge remote-tracking branch 'origin/master' into lexnv/storage-diff
751d8ff
@lexnv
archive: Remove variable
3c2bb7c 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
Update cargo lock
8f8c0c0 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive: Mode deduplicate_storage_diff_items to inenr methods
07a459c 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/tests: Ensure other storage entries are not provided
a151bf3 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/diff: Implement lexicographic_diff
a575b4a 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
archive/tests: Check lexicographic diff
caa75cc 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv
Merge remote-tracking branch 'origin/master' into lexnv/storage-diff
7ff0bdb
@lexnv
archive: Remove commented code and add documentation
c6e40a0 
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv @jsdw
Update substrate/client/rpc-spec-v2/src/archive/api.rs
a07b5c4 
Co-authored-by: James Wilson <[email protected]>
@paritytech-workflow-stopper
Copy link

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/12053348230
Failed job name: cargo-clippy

@lexnv lexnv added this pull request to the merge queue Nov 27, 2024
Merged via the queue into master with commit afd065f Nov 27, 2024
194 of 198 checks passed
@lexnv lexnv deleted the lexnv/storage-diff branch November 27, 2024 19:04
github-merge-queue bot pushed a commit that referenced this pull request Nov 29, 2024
@lexnv @jsdw
archive: Refactor archive_storage method into subscription (#6483)
4e7c968 
This PR adapts the `archive_storage` implementation from a method to a
subscription.

This keeps the archive APIs uniform and consistent.

Builds on: #5997

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
lexnv added a commit that referenced this pull request Nov 29, 2024
@lexnv @jsdw
rpc-v2: Implement archive_unstable_storageDiff (#5997)
4d350e5 
This PR implements the `archive_unstable_storageDiff`.

The implementation follows the rpc-v2 spec from:
-  paritytech/json-rpc-interface-spec#159.
- builds on top of
paritytech/json-rpc-interface-spec#161

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
Signed-off-by: Alexandru Vasile <[email protected]>
@lexnv lexnv mentioned this pull request Nov 29, 2024
Merged
lexnv added a commit that referenced this pull request Nov 29, 2024
@lexnv @jsdw
archive: Refactor archive_storage method into subscription (#6483)
53b0729 
This PR adapts the `archive_storage` implementation from a method to a
subscription.

This keeps the archive APIs uniform and consistent.

Builds on: #5997

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
EgorPopelyaev pushed a commit that referenced this pull request Dec 3, 2024
@lexnv @jsdw
backport/rpc-v2: Implement archive_unstable_storageDiff (stable2412) (
919663c 
#6709)

Backport for: #5997

cc @paritytech/subxt-team

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
lexnv added a commit that referenced this pull request Dec 3, 2024
@lexnv @jsdw
archive: Refactor archive_storage method into subscription (#6483)
1411929 
This PR adapts the `archive_storage` implementation from a method to a
subscription.

This keeps the archive APIs uniform and consistent.

Builds on: #5997

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
Krayt78 pushed a commit to Krayt78/polkadot-sdk that referenced this pull request Dec 18, 2024
@lexnv @jsdw @Krayt78
rpc-v2: Implement archive_unstable_storageDiff (paritytech#5997)
a7bc763 
This PR implements the `archive_unstable_storageDiff`.

The implementation follows the rpc-v2 spec from:
-  paritytech/json-rpc-interface-spec#159.
- builds on top of
paritytech/json-rpc-interface-spec#161

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
Krayt78 pushed a commit to Krayt78/polkadot-sdk that referenced this pull request Dec 18, 2024
@lexnv @jsdw @Krayt78
archive: Refactor archive_storage method into subscription (parityt…
0e673c3 
…ech#6483)

This PR adapts the `archive_storage` implementation from a method to a
subscription.

This keeps the archive APIs uniform and consistent.

Builds on: paritytech#5997

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
dudo50 pushed a commit to paraspell-research/polkadot-sdk that referenced this pull request Jan 4, 2025
@lexnv @jsdw @dudo50
rpc-v2: Implement archive_unstable_storageDiff (paritytech#5997)
f9e79cb 
This PR implements the `archive_unstable_storageDiff`.

The implementation follows the rpc-v2 spec from:
-  paritytech/json-rpc-interface-spec#159.
- builds on top of
paritytech/json-rpc-interface-spec#161

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
dudo50 pushed a commit to paraspell-research/polkadot-sdk that referenced this pull request Jan 4, 2025
@lexnv @jsdw @dudo50
archive: Refactor archive_storage method into subscription (parityt…
314745c 
…ech#6483)

This PR adapts the `archive_storage` implementation from a method to a
subscription.

This keeps the archive APIs uniform and consistent.

Builds on: paritytech#5997

cc @paritytech/subxt-team

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@niklasad1 niklasad1 niklasad1 approved these changes

@jsdw jsdw jsdw approved these changes

Assignees

@lexnv lexnv

Labels
D1-medium Can be fixed by a coder with good Rust knowledge but little knowledge of the codebase. I5-enhancement An additional feature request. T3-RPC_API This PR/Issue is related to RPC APIs.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lexnv @jsdw @niklasad1