Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage refactoring new hope for review #1331

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
2b26752
Import new unstable functions with transparent hashing.
xgreenx Jul 21, 2022
8dbf040
primitives crate:
xgreenx Jul 21, 2022
341c48b
metadata crate:
xgreenx Jul 21, 2022
346529c
Removed `initialize_contract` and related to initialization stuff. No…
xgreenx Jul 21, 2022
b731f92
Removed the old codegen related to spread and packed layout. If some …
xgreenx Jul 21, 2022
d6db8de
Updated all examples to use a new API.
xgreenx Jul 21, 2022
f967986
UI tests for a new codegen.
xgreenx Jul 21, 2022
fc3fcb0
Merge branch 'master' into feature/storage-rework-new-hope
xgreenx Aug 15, 2022
8f511ec
Apply all suggestion from the review
xgreenx Aug 15, 2022
7e094eb
Make CI happy
xgreenx Aug 15, 2022
8c9942d
Fix tests
xgreenx Aug 15, 2022
9188ca7
Fix tests
xgreenx Aug 15, 2022
7375b4c
Fix tests
xgreenx Aug 15, 2022
906b1d8
Fix tests
xgreenx Aug 15, 2022
290b384
Apply suggestions:
xgreenx Aug 16, 2022
1f7603f
Fix doc
xgreenx Aug 16, 2022
09c24e1
Merge branch 'master' into feature/storage-rework-new-hope
xgreenx Aug 16, 2022
07a71ea
Add comment to autoref specialisation
xgreenx Aug 16, 2022
07889af
Suggestion from the review
xgreenx Aug 17, 2022
f64d7f7
Revert back u8
xgreenx Aug 17, 2022
efff8bf
Remove unwrap
xgreenx Aug 17, 2022
556ce20
Collapse if let
xgreenx Aug 17, 2022
5ce970d
Fixed overflow for enums
xgreenx Aug 17, 2022
8fcdab2
Fixing comments
xgreenx Aug 18, 2022
ca9f95b
Renamed `Item` to `StorableHint` and `AutoItem` to `AutoStorableHint`
xgreenx Aug 19, 2022
8eb4cb9
Fix test
xgreenx Aug 19, 2022
66a6901
Renamed key_holder.
xgreenx Aug 22, 2022
8854be8
Nightly fmt
xgreenx Aug 22, 2022
15563de
Remove `Packed` path
xgreenx Aug 22, 2022
4f35230
Fix doc test
xgreenx Aug 22, 2022
5065b07
Apply suggestions from hte review
xgreenx Aug 26, 2022
3247ee2
Merge branch 'master' into feature/storage-rework-new-hope
xgreenx Aug 26, 2022
63e3909
Fixed build
xgreenx Aug 26, 2022
c07b471
Fix build
xgreenx Aug 26, 2022
72c84a8
Removed `initialize_contract` from linting and deleted all tests
xgreenx Aug 26, 2022
cad9234
Fix doc link
xgreenx Aug 26, 2022
7371905
Merge branch 'master' into feature/storage-rework-new-hope
xgreenx Aug 29, 2022
477e4cf
Fix mapping example
xgreenx Aug 30, 2022
235d649
Applied suggestion.
xgreenx Aug 31, 2022
8b9ccbc
Removed `delegate-calls` from the CI. Replaced it with `set-code-hash`
xgreenx Aug 31, 2022
a42bc9d
Merge remote-tracking branch 'parity/master' into feature/storage-rew…
xgreenx Aug 31, 2022
be1563e
fix test
xgreenx Aug 31, 2022
300b139
fix test
xgreenx Aug 31, 2022
7b8733d
Fix CI to use stable for contract build
xgreenx Aug 31, 2022
3e3fbd7
Fix CI to use stable for examples
xgreenx Aug 31, 2022
5808157
Merge remote-tracking branch 'parity/master' into feature/storage-rew…
xgreenx Sep 1, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .config/cargo_spellcheck.dic
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ defragmentation
deploy
dereferencing
deserialize/S
deserialization
dispatchable/S
encodable
evaluable
Expand Down Expand Up @@ -106,3 +107,4 @@ natively
payability
unpayable
initializer
storable
15 changes: 10 additions & 5 deletions .github/workflows/examples.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,20 @@ jobs:
# Once this is resolved we should re-enable building the examples on windows
# - windows-latest
toolchain:
- nightly
- stable
job:
- build
- test
runs-on: ${{ matrix.platform }}
env:
UPGRADEABLE_CONTRACTS: "forward-calls delegate-calls"
UPGRADEABLE_CONTRACTS: "forward-calls set-code-hash"
DELEGATOR_SUBCONTRACTS: "accumulator adder subber"
RUST_BACKTRACE: full
# We need to enable `RUSTC_BOOTSTRAP` so that the nightly ink!
# features still work on stable. This is done automatically by
# `cargo-contract`, but in our CI here we also use e.g.
# `cargo check` directly.
RUSTC_BOOTSTRAP: "1"
steps:

- uses: actions/setup-node@v3
Expand Down Expand Up @@ -90,12 +95,12 @@ jobs:
echo "Processing delegator contract: $contract";
cargo contract ${{ matrix.job }} --verbose --manifest-path examples/delegator/${contract}/Cargo.toml;
}
$upgradeable_contracts = "forward-calls","delegate-calls"
$upgradeable_contracts = "forward-calls","set-code-hash"
foreach ($contract in $upgradeable_contracts) {
echo "Processing upgradeable contract: $contract";
cargo contract ${{ matrix.job }} --verbose --manifest-path examples/upgradeable-contracts/${contract}/Cargo.toml;
}
cargo contract ${{ matrix.job }} --verbose --manifest-path examples/upgradeable-contracts/delegate-calls/upgradeable-flipper/Cargo.toml;
cargo contract ${{ matrix.job }} --verbose --manifest-path examples/upgradeable-contracts/set-code-hash/updated-incrementer/Cargo.toml;
foreach ($example in Get-ChildItem -Directory "examples\*") {
if ($example -Match 'upgradeable-contracts') { continue }
echo "Processing example: $example";
Expand All @@ -114,7 +119,7 @@ jobs:
echo "Processing upgradeable contract: $contract";
cargo contract ${{ matrix.job }} --verbose --manifest-path=examples/upgradeable-contracts/$contract/Cargo.toml;
done
cargo contract ${{ matrix.job }} --verbose --manifest-path=examples/upgradeable-contracts/delegate-calls/upgradeable-flipper/Cargo.toml;
cargo contract ${{ matrix.job }} --verbose --manifest-path=examples/upgradeable-contracts/set-code-hash/updated-incrementer/Cargo.toml;
for example in examples/*/; do
if [ "$example" = "examples/upgradeable-contracts/" ]; then continue; fi;
echo "Processing example: $example";
Expand Down
16 changes: 8 additions & 8 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ variables:
ALSO_WASM_CRATES: "env storage storage/derive allocator prelude primitives lang lang/macro lang/ir"
ALL_CRATES: "${PURELY_STD_CRATES} ${ALSO_WASM_CRATES}"
DELEGATOR_SUBCONTRACTS: "accumulator adder subber"
UPGRADEABLE_CONTRACTS: "forward-calls delegate-calls"
UPGRADEABLE_CONTRACTS: "forward-calls set-code-hash"
# We need to enable `RUSTC_BOOTSTRAP` so that the nightly ink!
# features still work on stable. This is done automatically by
# `cargo-contract`, but in our CI here we also use e.g.
Expand Down Expand Up @@ -127,7 +127,7 @@ examples-fmt:
- for contract in ${UPGRADEABLE_CONTRACTS}; do
cargo +nightly fmt --verbose --manifest-path ./examples/upgradeable-contracts/${contract}/Cargo.toml -- --check;
done
- cargo +nightly fmt --verbose --manifest-path ./examples/upgradeable-contracts/delegate-calls/upgradeable-flipper/Cargo.toml -- --check
- cargo +nightly fmt --verbose --manifest-path ./examples/upgradeable-contracts/set-code-hash/updated-incrementer/Cargo.toml -- --check
allow_failure: true

clippy-std:
Expand Down Expand Up @@ -164,7 +164,7 @@ examples-clippy-std:
- for contract in ${UPGRADEABLE_CONTRACTS}; do
cargo +nightly clippy --verbose --all-targets --manifest-path ./examples/upgradeable-contracts/${contract}/Cargo.toml -- -D warnings;
done
- cargo +nightly clippy --verbose --all-targets --manifest-path ./examples/upgradeable-contracts/delegate-calls/upgradeable-flipper/Cargo.toml -- -D warnings;
- cargo +nightly clippy --verbose --all-targets --manifest-path ./examples/upgradeable-contracts/set-code-hash/updated-incrementer/Cargo.toml -- -D warnings;
allow_failure: true

examples-clippy-wasm:
Expand All @@ -182,7 +182,7 @@ examples-clippy-wasm:
- for contract in ${UPGRADEABLE_CONTRACTS}; do
cargo +nightly clippy --verbose --manifest-path ./examples/upgradeable-contracts/${contract}/Cargo.toml --no-default-features --target wasm32-unknown-unknown -- -D warnings;
done
- cargo +nightly clippy --verbose --manifest-path ./examples/upgradeable-contracts/delegate-calls/upgradeable-flipper/Cargo.toml --no-default-features --target wasm32-unknown-unknown -- -D warnings;
- cargo +nightly clippy --verbose --manifest-path ./examples/upgradeable-contracts/set-code-hash/updated-incrementer/Cargo.toml --no-default-features --target wasm32-unknown-unknown -- -D warnings;
allow_failure: true


Expand Down Expand Up @@ -281,7 +281,7 @@ docs:
- cargo doc --no-deps --all-features
-p scale-info -p ink_metadata
-p ink_env -p ink_storage -p ink_storage_derive
-p ink_primitives -p ink_prelude
-p ink_primitives -p ink_prelude -p ink_primitives_derive
-p ink_lang -p ink_lang_macro -p ink_lang_ir -p ink_lang_codegen
- mv ${CARGO_TARGET_DIR}/doc ./crate-docs
# FIXME: remove me after CI image gets nonroot
Expand Down Expand Up @@ -352,7 +352,7 @@ examples-test:
- for contract in ${UPGRADEABLE_CONTRACTS}; do
cargo test --verbose --manifest-path ./examples/upgradeable-contracts/${contract}/Cargo.toml;
done
- cargo test --verbose --manifest-path ./examples/upgradeable-contracts/delegate-calls/upgradeable-flipper/Cargo.toml;
- cargo test --verbose --manifest-path ./examples/upgradeable-contracts/set-code-hash/updated-incrementer/Cargo.toml;

examples-contract-build:
stage: examples
Expand All @@ -372,7 +372,7 @@ examples-contract-build:
- for contract in ${UPGRADEABLE_CONTRACTS}; do
cargo +stable contract build --manifest-path ./examples/upgradeable-contracts/${contract}/Cargo.toml;
done
- cargo +stable contract build --manifest-path ./examples/upgradeable-contracts/delegate-calls/upgradeable-flipper/Cargo.toml
- cargo +stable contract build --manifest-path ./examples/upgradeable-contracts/set-code-hash/updated-incrementer/Cargo.toml

examples-docs:
stage: examples
Expand All @@ -395,7 +395,7 @@ examples-docs:
- for contract in ${UPGRADEABLE_CONTRACTS}; do
cargo doc --manifest-path ./examples/upgradeable-contracts/${contract}/Cargo.toml --document-private-items --verbose --no-deps;
done
- cargo doc --manifest-path ./examples/upgradeable-contracts/delegate-calls/upgradeable-flipper/Cargo.toml --document-private-items --verbose --no-deps
- cargo doc --manifest-path ./examples/upgradeable-contracts/set-code-hash/updated-incrementer/Cargo.toml --document-private-items --verbose --no-deps


#### stage: ink-waterfall
Expand Down
13 changes: 8 additions & 5 deletions crates/engine/src/ext.rs
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ impl Engine {

/// Writes the encoded value into the storage at the given key.
/// Returns the size of the previously stored value at the key if any.
pub fn set_storage(&mut self, key: &[u8; 32], encoded_value: &[u8]) -> Option<u32> {
pub fn set_storage(&mut self, key: &[u8], encoded_value: &[u8]) -> Option<u32> {
let callee = self.get_callee();
let account_id = AccountId::from_bytes(&callee[..]);

Expand All @@ -243,7 +243,7 @@ impl Engine {
}

/// Returns the decoded contract storage at the key if any.
pub fn get_storage(&mut self, key: &[u8; 32], output: &mut &mut [u8]) -> Result {
pub fn get_storage(&mut self, key: &[u8], output: &mut &mut [u8]) -> Result {
let callee = self.get_callee();
let account_id = AccountId::from_bytes(&callee[..]);

Expand All @@ -258,7 +258,7 @@ impl Engine {
}

/// Returns the size of the value stored in the contract storage at the key if any.
pub fn contains_storage(&mut self, key: &[u8; 32]) -> Option<u32> {
pub fn contains_storage(&mut self, key: &[u8]) -> Option<u32> {
let callee = self.get_callee();
let account_id = AccountId::from_bytes(&callee[..]);

Expand All @@ -269,14 +269,17 @@ impl Engine {
}

/// Removes the storage entries at the given key.
ascjones marked this conversation as resolved.
Show resolved Hide resolved
pub fn clear_storage(&mut self, key: &[u8; 32]) {
/// Returns the size of the previously stored value at the key if any.
pub fn clear_storage(&mut self, key: &[u8]) -> Option<u32> {
let callee = self.get_callee();
let account_id = AccountId::from_bytes(&callee[..]);
self.debug_info.inc_writes(account_id.clone());
let _ = self
.debug_info
.remove_cell_for_account(account_id, key.to_vec());
let _ = self.database.remove_contract_storage(&callee, key);
self.database
.remove_contract_storage(&callee, key)
.map(|val| val.len() as u32)
}

/// Remove the calling account and transfer remaining balance.
Expand Down
43 changes: 26 additions & 17 deletions crates/env/src/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ use crate::{
Environment,
Result,
};
use ink_primitives::Key;
use ink_primitives::traits::Storable;

/// Returns the address of the caller of the executed contract.
///
Expand Down Expand Up @@ -183,49 +183,58 @@ where
})
}

/// Writes the value to the contract storage under the given key and returns
/// the size of pre-existing value at the specified key if any.
/// Writes the value to the contract storage under the given storage key and returns the size
/// of pre-existing value if any.
///
/// # Panics
///
/// - If the encode length of value exceeds the configured maximum value length of a storage entry.
pub fn set_contract_storage<V>(key: &Key, value: &V) -> Option<u32>
pub fn set_contract_storage<K, V>(key: &K, value: &V) -> Option<u32>
where
V: scale::Encode,
K: scale::Encode,
V: Storable,
{
<EnvInstance as OnInstance>::on_instance(|instance| {
EnvBackend::set_contract_storage::<V>(instance, key, value)
EnvBackend::set_contract_storage::<K, V>(instance, key, value)
})
}

/// Returns the value stored under the given key in the contract's storage if any.
/// Returns the value stored under the given storage key in the contract's storage if any.
///
/// # Errors
///
/// - If the decoding of the typed value failed (`KeyNotFound`)
pub fn get_contract_storage<R>(key: &Key) -> Result<Option<R>>
pub fn get_contract_storage<K, R>(key: &K) -> Result<Option<R>>
where
R: scale::Decode,
K: scale::Encode,
R: Storable,
{
<EnvInstance as OnInstance>::on_instance(|instance| {
EnvBackend::get_contract_storage::<R>(instance, key)
EnvBackend::get_contract_storage::<K, R>(instance, key)
})
}

/// Checks whether there is a value stored under the given key in
/// the contract's storage.
/// Checks whether there is a value stored under the given storage key in the contract's storage.
///
/// If a value is stored under the specified key, the size of the value is returned.
pub fn contract_storage_contains(key: &Key) -> Option<u32> {
pub fn contains_contract_storage<K>(key: &K) -> Option<u32>
where
K: scale::Encode,
{
<EnvInstance as OnInstance>::on_instance(|instance| {
EnvBackend::contract_storage_contains(instance, key)
EnvBackend::contains_contract_storage::<K>(instance, key)
})
}

/// Clears the contract's storage key entry.
pub fn clear_contract_storage(key: &Key) {
/// Clears the contract's storage entry under the given storage key.
ascjones marked this conversation as resolved.
Show resolved Hide resolved
///
/// If a value was stored under the specified storage key, the size of the value is returned.
pub fn clear_contract_storage<K>(key: &K) -> Option<u32>
where
K: scale::Encode,
{
<EnvInstance as OnInstance>::on_instance(|instance| {
EnvBackend::clear_contract_storage(instance, key)
EnvBackend::clear_contract_storage::<K>(instance, key)
})
}

Expand Down
33 changes: 21 additions & 12 deletions crates/env/src/backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ use crate::{
Environment,
Result,
};
use ink_primitives::Key;
use ink_primitives::traits::Storable;

/// The flags to indicate further information about the end of a contract execution.
#[derive(Default)]
Expand Down Expand Up @@ -162,26 +162,35 @@ impl CallFlags {

/// Environmental contract functionality that does not require `Environment`.
pub trait EnvBackend {
/// Writes the value to the contract storage under the given key and returns
/// the size of the pre-existing value at the specified key if any.
fn set_contract_storage<V>(&mut self, key: &Key, value: &V) -> Option<u32>
/// Writes the value to the contract storage under the given storage key.
///
/// Returns the size of the pre-existing value at the specified key if any.
fn set_contract_storage<K, V>(&mut self, key: &K, value: &V) -> Option<u32>
where
V: scale::Encode;
K: scale::Encode,
V: Storable;

/// Returns the value stored under the given key in the contract's storage if any.
/// Returns the value stored under the given storage key in the contract's storage if any.
///
/// # Errors
///
/// - If the decoding of the typed value failed
fn get_contract_storage<R>(&mut self, key: &Key) -> Result<Option<R>>
fn get_contract_storage<K, R>(&mut self, key: &K) -> Result<Option<R>>
where
R: scale::Decode;
K: scale::Encode,
R: Storable;

/// Returns the size of a value stored under the specified key is returned if any.
fn contract_storage_contains(&mut self, key: &Key) -> Option<u32>;
/// Returns the size of a value stored under the given storage key is returned if any.
fn contains_contract_storage<K>(&mut self, key: &K) -> Option<u32>
where
K: scale::Encode;

/// Clears the contract's storage key entry.
fn clear_contract_storage(&mut self, key: &Key);
/// Clears the contract's storage key entry under the given storage key.
ascjones marked this conversation as resolved.
Show resolved Hide resolved
///
/// Returns the size of the previously stored value at the specified key if any.
fn clear_contract_storage<K>(&mut self, key: &K) -> Option<u32>
where
K: scale::Encode;

/// Returns the execution input to the executed contract and decodes it as `T`.
///
Expand Down
35 changes: 22 additions & 13 deletions crates/env/src/engine/off_chain/impls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ use ink_engine::{
ext,
ext::Engine,
};
use ink_primitives::Key;
use ink_primitives::traits::Storable;

/// The capacity of the static buffer.
/// This is the same size as the ink! on-chain environment. We chose to use the same size
Expand Down Expand Up @@ -184,34 +184,43 @@ impl EnvInstance {
}

impl EnvBackend for EnvInstance {
fn set_contract_storage<V>(&mut self, key: &Key, value: &V) -> Option<u32>
fn set_contract_storage<K, V>(&mut self, key: &K, value: &V) -> Option<u32>
where
V: scale::Encode,
K: scale::Encode,
V: Storable,
{
let v = scale::Encode::encode(value);
self.engine.set_storage(key.as_ref(), &v[..])
let mut v = vec![];
Storable::encode(value, &mut v);
self.engine.set_storage(&key.encode(), &v[..])
}

fn get_contract_storage<R>(&mut self, key: &Key) -> Result<Option<R>>
fn get_contract_storage<K, R>(&mut self, key: &K) -> Result<Option<R>>
where
R: scale::Decode,
K: scale::Encode,
R: Storable,
{
let mut output: [u8; 9600] = [0; 9600];
match self.engine.get_storage(key.as_ref(), &mut &mut output[..]) {
match self.engine.get_storage(&key.encode(), &mut &mut output[..]) {
Ok(_) => (),
Err(ext::Error::KeyNotFound) => return Ok(None),
Err(_) => panic!("encountered unexpected error"),
}
let decoded = scale::Decode::decode(&mut &output[..])?;
let decoded = Storable::decode(&mut &output[..])?;
Ok(Some(decoded))
}

fn contract_storage_contains(&mut self, key: &Key) -> Option<u32> {
self.engine.contains_storage(key.as_ref())
fn contains_contract_storage<K>(&mut self, key: &K) -> Option<u32>
where
K: scale::Encode,
{
self.engine.contains_storage(&key.encode())
}

fn clear_contract_storage(&mut self, key: &Key) {
self.engine.clear_storage(key.as_ref())
fn clear_contract_storage<K>(&mut self, key: &K) -> Option<u32>
where
K: scale::Encode,
{
self.engine.clear_storage(&key.encode())
}

fn decode_input<T>(&mut self) -> Result<T>
Expand Down
Loading