Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Adds export-state subcommand #5842

Merged
merged 7 commits into from
Apr 30, 2020
Merged

Adds export-state subcommand #5842

merged 7 commits into from
Apr 30, 2020

Conversation

bkchr
Copy link
Member

@bkchr bkchr commented Apr 30, 2020

This adds a new sub-command export-state. This sub-command can be used to export the state of a running chain as a chain spec. This is useful for testing runtime upgrades before distributing them on a live chain.

It basically collects all the key value pairs at a given block and overwrites the genesis storage values with the collected ones. The build chain-spec will not work right away, because Babe will complain about unexpected epoch changes and the authority set will contain authorities for which the user doesn't have the key. This will be solved by an extra tool that replaces the corresponding keys and makes the chain spec usable as new a new genesis.

@bkchr bkchr added A0-please_review Pull request needs code review. B1-clientnoteworthy labels Apr 30, 2020
@bkchr bkchr requested a review from cecton as a code owner April 30, 2020 10:05

// Remove all default child storage roots from the top storage and collect the child storage
// pairs.
while let Some(pos) = top_storage
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cheme it would be nice if you could take a look here, if I have done this child value collection correctly.

@arkpar
Copy link
Member

arkpar commented Apr 30, 2020

Looks good, except it loads all of the state into memory. This is fine for now, but might cause problems with bigger chains in the future.
Ideally it should be writing the file while iterating over the state.

@bkchr
Copy link
Member Author

bkchr commented Apr 30, 2020

Looks good, except it loads all of the state into memory. This is fine for now, but might cause problems with bigger chains in the future.
Ideally it should be writing the file while iterating over the state.

Yeah, the whole operation isn't that nice, but as this is only some sort of debug command. I thought that we can live with the high memory load. ;)

Ok::<_, Error>(())
})?;

children_default.insert(key.0, StorageChild { child_info, data: pairs });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we should insert with key (without prefix), children_default works on unprefixed key.
If only use at

if storage.top.iter().any(|(k, _)| well_known_keys::is_child_storage_key(k)) {
it is ignored, but could be an issue later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I m stoopid, key was redefined with the unprefixed one so it is the unprefixed one, so please forget previous comment.
Algo looks correct to 👍

@@ -148,6 +148,9 @@ pub mod well_known_keys {
/// Prefix of child storage keys.
pub const CHILD_STORAGE_KEY_PREFIX: &'static [u8] = b":child_storage:";

/// Prefix of the default child storage keys in the top trie.
pub const DEFAULT_CHILD_STORAGE_KEY_PREFIX: &'static [u8] = b":child_storage:default:";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally this should not be made public, we would use

let (type, key) = ChildType::from_prefixed_key(prefixed_key);
assert!(type == ChildType::Default);

);
let child_info = ChildInfo::new_default(&key.0);

let keys = self.client.child_storage_keys(&block, &child_info, &key)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here third parameter should be empty prefix not key.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find ;)

client/service/src/chain_ops.rs Outdated Show resolved Hide resolved
client/service/src/chain_ops.rs Outdated Show resolved Hide resolved
);
let child_info = ChildInfo::new_default(&key.0);

let keys = self.client.child_storage_keys(&block, &child_info, &key)?;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good find ;)

client/service/src/chain_ops.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@cecton cecton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLI is 👌

use structopt::StructOpt;

/// The `check-block` command used to validate blocks.
#[derive(Debug, StructOpt, Clone)]
pub struct CheckBlockCmd {
/// Block hash or number
#[structopt(value_name = "HASH or NUMBER")]
pub input: String,
pub input: BlockNumberOrHash,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet!

2 + pos,
))
} else {
Ok(Self(block_number.into()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be probably handy if we had a try_into for this

@gavofyork gavofyork merged commit bb83628 into master Apr 30, 2020
@gavofyork gavofyork deleted the bkchr-export-state-cli branch April 30, 2020 13:44
@xlc
Copy link
Contributor

xlc commented Apr 30, 2020

Nice. Where can I find this "extra tool that replaces the corresponding keys and makes the chain spec usable as new a new genesis" ?

I always wants a feature to fork a live network and do some experiments on them. Super useful when working with ethereum.

@bkchr
Copy link
Member Author

bkchr commented May 1, 2020

This does not exist yet, I will work on this in the next time.

@xcaptain
Copy link
Contributor

@bkchr I use export-state to create a new genesis file and use --chain state_file.json to start a new chain, I also got the ClientImport(expected epoch change) error, is that because my old keys changed?

@bkchr
Copy link
Member Author

bkchr commented Dec 14, 2021

No, you will need to remove the epoch related storage keys from the exported genesis config to make babe work.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A0-please_review Pull request needs code review.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants