diff --git a/docs/advanced/datastructures/custom.md b/docs/advanced/datastructures/custom.md index babaac65f4..be1ded0554 100644 --- a/docs/advanced/datastructures/custom.md +++ b/docs/advanced/datastructures/custom.md @@ -13,17 +13,17 @@ manipulate the contract's storage. However, contract authors should know that th also create their own custom data structures. ## Using custom types on storage -Any custom type wanting to be compatible with ink! storage must implement the -[`Storable`](https://use-ink.github.io/ink/ink_storage_traits/trait.Storable.html) -trait, so it can be SCALE -[`encoded`](https://docs.rs/parity-scale-codec/latest/parity_scale_codec/trait.Encode.html) -and -[`decoded`](https://docs.rs/parity-scale-codec/latest/parity_scale_codec/trait.Decode.html). -Additionally, the traits -[`StorageLayout`](https://use-ink.github.io/ink/ink_storage_traits//trait.StorageLayout.html) -and [`TypeInfo`](https://docs.rs/scale-info/latest/scale_info/trait.TypeInfo.html) +Any custom type wanting to be compatible with ink! storage must implement the [`Storable`][Storable] trait, +so it can be SCALE [`encoded`][scale-encode] and [`decoded`][scale-decode]. +Additionally, the traits [`StorageLayout`][StorageLayout] and [`TypeInfo`][TypeInfo] are required as well. But don't worry, usually these traits can just be derived: +[Storable]: https://use-ink.github.io/ink/ink_storage_traits/trait.Storable.html +[scale-encode]: https://docs.rs/parity-scale-codec/latest/parity_scale_codec/trait.Encode.html +[scale-decode]: https://docs.rs/parity-scale-codec/latest/parity_scale_codec/trait.Decode.html +[StorageLayout]: https://use-ink.github.io/ink/ink_storage_traits/trait.StorageLayout.html +[TypeInfo]: https://docs.rs/scale-info/latest/scale_info/trait.TypeInfo.html + ```rust /// A custom type that we can use in our contract storage #[ink::scale_derive(Encode, Decode, TypeInfo)] @@ -41,11 +41,12 @@ pub struct ContractStorage { } ``` -Even better: there is a macro -[`#[ink::storage_item]`](https://use-ink.github.io/ink/ink_macro/attr.storage_item.html), +Even better: there is a macro [`#[ink::storage_item]`][storage_item], which derives all necessary traits for you. If there is no need to implement any special behavior, the above code example can be simplified further as follows: +[storage_item]: https://use-ink.github.io/ink/ink_macro/attr.storage_item.html + ```rust /// A custom type that we can use in our contract storage #[ink::storage_item] @@ -65,10 +66,10 @@ the relevant trait documentations for more information. :::note The `#[ink::storage_item]` macro is responsible for storage key calculation of -non-[`Packed`](https://use-ink.github.io/ink/ink_storage_traits//trait.Packed.html) +non-[`Packed`](https://use-ink.github.io/ink/ink_storage_traits/trait.Packed.html) types. Without it, the key for non-`Packed` fields will be zero. Using this macro is necessary if you don't plan to use a -[`ManualKey`](https://use-ink.github.io/ink/ink_storage_traits//struct.ManualKey.html) +[`ManualKey`](https://use-ink.github.io/ink/ink_storage_traits/struct.ManualKey.html) on a non-`Packed` type. Types with custom implementations of the ink! storage traits can still use this macro only @@ -76,6 +77,32 @@ for key calculation by disabling the derives: `#[ink::storage_item(derive = fals ::: +:::note +The `#[ink::storage_item]` macro defaults to deriving a "non-Packed" layout +(i.e. the derived storage type doesn't implement the [`Packed`][Packed] trait by default). +So if the intention is to use the custom storage type in a position that expects +a packed storage type (e.g. as `V` in `Mapping` or `T` in `Vec`), +you can use the `packed` argument/flag to derive an implementation of the `Packed` trait +e.g. + +```rust +/// A `Packed` custom storage type that can be used as a `T: Packed` type +/// (e.g. as `V` in `Mapping` or `T` in `Vec`). +#[ink::storage_item(packed)] +pub struct Inner { + value: bool, +} + +#[ink(storage)] +pub struct ContractStorage { + inner: Mapping, +} + +``` +::: + +[Packed]: https://use-ink.github.io/ink/ink_storage_traits/trait.Packed.html + ## Generic storage fields It is possible to use generic data types in your storage, as long as any generic type diff --git a/docs/advanced/datastructures/overview.md b/docs/advanced/datastructures/overview.md index 6be57659d3..228b124f70 100644 --- a/docs/advanced/datastructures/overview.md +++ b/docs/advanced/datastructures/overview.md @@ -9,20 +9,24 @@ hide_title: true # Overview The `ink_storage` crate acts as the standard storage library for ink! smart contracts. -At the moment it provides two primitives for interacting with storage, -[`Mapping`](https://use-ink.github.io/ink/ink_storage/struct.Mapping.html) -and [`Lazy`](https://use-ink.github.io/ink/ink_storage/struct.Lazy.html). +At the moment it provides three primitives for interacting with storage, +[`Mapping`][mapping], [`Lazy`][lazy] and [`StorageVec`][storage-vec]. -`Mapping` is a mapping of key-value pairs directly to the contract storage. It is very -similar to traditional hash tables and comparable to the `mapping` type Solidity offers. +[mapping]: https://use-ink.github.io/ink/ink_storage/struct.Mapping.html +[lazy]: https://use-ink.github.io/ink/ink_storage/struct.Lazy.html +[storage-vec]: https://use-ink.github.io/ink/ink_storage/struct.StorageVec.html + +[`Mapping`][mapping] is a mapping of key-value pairs directly to the contract storage. +It is very similar to traditional hash tables and comparable to the `mapping` type Solidity offers. As a core ingredient to the ink! language, its main advantage is being simple and lightweight: It favors being efficient in terms of gas costs and code size over providing a lot of high-level functionality found in other implementations like the `ink::prelude::collections::HashMap` type. -Overall, the ink! `Mapping` will be a solid choice for most contracts. Moreover, smart -contracts developers can implement advanced features themselves. +Overall, the ink! `Mapping` will be a solid choice for most contracts. +Moreover, smart contracts developers can implement advanced features themselves. +You can learn more about this in the [dedicated `Mapping` section](./mapping.md). -`Lazy` is a wrapper type that can be used over any other storage compatible type. +[`Lazy`][lazy] is a wrapper type that can be used over any other storage compatible type. This allows smart contract developers fine-grained manual control over the layout of the contract storage by assigning a separate storage cell for the field. For example, it can be used to prevent the contract from eagerly loading large storage fields @@ -30,3 +34,10 @@ during each contract call. Conceivably, it may be desirable to change certain aspects on how your contract deals with its storage variables. You can find out more about this in the section about the ink! [Storage Layout](./storage-layout.md). + +[`StorageVec`][storage-vec] is a vector of values/elements directly on contract storage. +However, unlike `Vec`, which stores all of its elements in a single storage cell, +`StorageVec` stores each of its elements in its own storage cell. +This layout is more gas efficient for reads and writes of single elements, +and also allows higher maximum capacity for `StorageVec` compared to `Vec`. +You can learn more about this in the [dedicated `StorageVec` section](./storagevec.md). diff --git a/docs/advanced/datastructures/storage-in-metadata.md b/docs/advanced/datastructures/storage-in-metadata.md index 99bc30de30..5c13801036 100644 --- a/docs/advanced/datastructures/storage-in-metadata.md +++ b/docs/advanced/datastructures/storage-in-metadata.md @@ -82,7 +82,7 @@ The storage will be reflected inside the metadata as like follows: We observe that the storage layout is represented as a tree, where tangible storage values end up inside a `leaf`. Because of -[`Packed`](https://use-ink.github.io/ink/ink_storage_traits//trait.Packed.html) +[`Packed`](https://use-ink.github.io/ink/ink_storage_traits/trait.Packed.html) encoding, leafs can share the same storage key, and in order to reach them you'd need to fetch and decode the whole storage cell under this key. diff --git a/docs/advanced/datastructures/storage-layout.md b/docs/advanced/datastructures/storage-layout.md index 1b923206b0..95b475b1a4 100644 --- a/docs/advanced/datastructures/storage-layout.md +++ b/docs/advanced/datastructures/storage-layout.md @@ -24,28 +24,30 @@ to ink! by the [`pallet-revive`](https://github.com/paritytech/polkadot-sdk/tree Storage Organization: Layout -Storage data is always encoded with the -[`SCALE`](https://docs.polkadot.com/polkadot-protocol/basics/data-encoding/#scale-codec-libraries) codec. -The storage API operates by storing and loading entries into and from a single storage -cells, where each storage cell is accessed under its own dedicated storage key. To some -extent, the storage API works similar to a traditional key-value database. +Storage data is always encoded with the [`SCALE`][SCALE] codec. +The storage API operates by storing and loading entries into and from single storage cells, +where each storage cell is accessed under its own dedicated storage key. +To some extent, the storage API works similar to a traditional key-value database. + +[SCALE]: https://docs.polkadot.com/polkadot-protocol/basics/data-encoding/#scale-codec-libraries ## Packed vs Non-Packed layout -Types that can be stored entirely under a single storage cell are considered -[`Packed`](https://use-ink.github.io/ink/ink_storage_traits//trait.Packed.html). +Types that can be stored entirely under a single storage cell are considered [`Packed`][Packed]. By default, ink! tries to store all storage struct fields under a single storage cell. Consequentially, with a `Packed` storage layout, any message interacting with the contract storage will always need to operate on the entire contract storage struct. +[Packed]: https://use-ink.github.io/ink/ink_storage_traits/trait.Packed.html + For example, if we have a somewhat small contract storage struct consisting of only a few tiny fields, pulling everything from the storage inside every message is not problematic. It may even be advantageous - especially if we expect most messages to interact with most of the storage fields. On the other hand, this can get problematic if we're storing a large `ink::prelude::vec::Vec` -in the contract storage but provide messages that do not need to read and write from this -`Vec`. In that scenario, each and every contract message bears runtime overhead by dealing +in the contract storage but provide messages that do not need to read and write from this `Vec`. +In that scenario, each and every contract message bears runtime overhead by dealing with that `Vec`, regardless whether they access it or not. This results in extra gas costs. To solve this problem we need to turn our storage into a non-packed layout somehow. @@ -55,23 +57,25 @@ To solve this problem we need to turn our storage into a non-packed layout someh :::caution -If any type exhibiting `Packed` layout gets large enough (an ever-growing `Vec` might be -a prime candidate for this), it will break your contract. -This is because for encoding and decoding storage items, there is a buffer with only limited -capacity (around 16KB in the default configuration) available. This means any contract -trying to decode more than that will trap! If you are unsure about the potential size a -data structure might get, consider using an ink! `Mapping`, which can store an arbitrary -number of elements, instead. +If any type exhibiting `Packed` layout gets large enough (an ever-growing `Vec` +might be a prime candidate for this), it will break your contract. +This is because for encoding and decoding storage items, there is a buffer +with only limited capacity (around 16KB in the default configuration) available. +This means any contract trying to decode more than that will trap! +If you are unsure about the potential size a data structure might get, consider using +an ink! `Mapping` or `StorageVec`, which can store an arbitrary number of elements, instead. ::: ## Eager Loading vs. Lazy Loading + ink! provides means of breaking the storage up into smaller pieces, which can be loaded -on demand, with the -[`Lazy`](https://use-ink.github.io/ink/ink/storage/struct.Lazy.html) primitive. -Wrapping any storage field inside a `Lazy` struct makes the storage -struct in which that field appears also -non-`Packed`, preventing it from being eagerly loaded during arbitrary storage operations: +on demand, with the [`Lazy`][Lazy] primitive. +Wrapping any storage field inside a `Lazy` struct makes the storage struct +in which that field appears also non-`Packed`, preventing it from being eagerly loaded +during arbitrary storage operations: + +[Lazy]: https://use-ink.github.io/ink/ink/storage/struct.Lazy.html
Storage Organization: Layout with a Lazy field @@ -127,22 +131,20 @@ mod mycontract { :::caution -`ink::prelude::vec::Vec`'s are always loaded in their entirety. This is because all elements -of the `ink::prelude::vec::Vec` live under a single storage key. Wrapping the -`ink::prelude::vec::Vec` inside `Lazy`, like the -provided example above does, has no influence on its inner layout. If you are dealing with -large or sparse arrays on contract storage, consider using a `Mapping` instead. +`ink::prelude::vec::Vec`'s are always loaded in their entirety. +This is because all elements of the `ink::prelude::vec::Vec` live under a single storage key. +Wrapping the `ink::prelude::vec::Vec` inside `Lazy`, like the provided example above does, +has no influence on its inner layout. If you are dealing with large or sparse arrays +on contract storage, consider using a `Mapping` instead. ::: ## Manual vs. Automatic Key Generation -By default, keys are calculated automatically for you, thanks to the -[`AutoKey`](https://use-ink.github.io/ink/ink_storage_traits//struct.AutoKey.html) -primitive. They'll be generated at compile time and ruled out for conflicts. -However, for non-`Packed` types like `Lazy` or the `Mapping`, the -[`ManualKey`](https://use-ink.github.io/ink/ink_storage_traits//struct.ManualKey.html) -primitive allows manual control over the storage key of a field like so: +By default, keys are calculated automatically for you, thanks to the [`AutoKey`][AutoKey] primitive. +They'll be generated at compile time and ruled out for conflicts. +However, for non-`Packed` types like `Lazy` or the `Mapping`, the [`ManualKey`][ManualKey] primitive +allows manual control over the storage key of a field like so: ```rust #[ink(storage)] @@ -156,6 +158,9 @@ This may be advantageous: Your storage key will always stay the same, regardless the version of your contract or ink! itself (note that the key calculation algorithm may change with future ink! versions). +[AutoKey]: https://use-ink.github.io/ink/ink_storage_traits/struct.AutoKey.html +[ManualKey]: https://use-ink.github.io/ink/ink_storage_traits/struct.ManualKey.html + :::tip Using `ManualKey` instead of `AutoKey` might be especially desirable for upgradable @@ -178,13 +183,15 @@ pub struct MyContract> { ## Considerations -It might be worthwhile to think about the desired storage layout of your contract. While -using a `Packed` layout will keep your contracts overall code size smaller, it can cause -unnecessarily high gas costs. Thus, we consider it a good practice to break up large -or complex storage layouts into reasonably sized distinct storage cells. +It might be worthwhile to think about the desired storage layout of your contract. +While using a `Packed` layout will keep your contracts overall code size smaller, +it can cause unnecessarily high gas costs. +Thus, we consider it a good practice to break up large or complex storage layouts +into reasonably sized distinct storage cells. :::note -ink! `Mapping`s are always non-`Packed` and loaded lazily, one key-value pair at the time. +ink! `Mapping`s and `StorageVec`s are always non-`Packed` and loaded lazily, +one key-value pair at the time. ::: diff --git a/docs/advanced/datastructures/storagevec.md b/docs/advanced/datastructures/storagevec.md index 6dfef110bf..2d1ae3736e 100644 --- a/docs/advanced/datastructures/storagevec.md +++ b/docs/advanced/datastructures/storagevec.md @@ -74,18 +74,22 @@ mod mycontract { ## Difference between `StorageVec` and Rusts `Vec` type -Any Rust `Vec` will exhibit `Packed` storage layout; where -`StorageVec` stores each value under it's own storage key. +Any Rust `Vec` will exhibit `Packed` storage layout +(i.e. all values are stored in a single storage cell); +whereas `StorageVec` stores each value under its own storage key. Hence, any read or write from or to a `Vec` on storage will load or store _all_ of its elements. -This can be undesirable: -The cost of reading or writing a _single_ element grows linearly -corresponding to the number of elements in the vector (its length). -Additionally, the maximum capacity of the _whole_ vector is limited by -the size of [ink!'s static buffer](https://github.com/use-ink/ink/blob/master/ARCHITECTURE.md#communication-with-the-pallet) -used during ABI encoding and decoding (default 16 KiB). +This can be undesirable because: +- The cost of reading or writing a _single_ element grows linearly + corresponding to the number of elements in the vector (i.e. its length). +- Additionally, the maximum capacity of the _whole_ vector is limited by + [`pallet-revive's` storage limit for a single cell][pallet-revive-limits] + (currently `416 bytes`). +- Lastly, the maximum capacity of the _whole_ vector is also limited by + the size of [ink!'s static buffer][static-buffer] used during ABI encoding + and decoding (current default is `16 KiB`). `StorageVec` on the other hand allows to access each element individually. Thus, it can theoretically grow to infinite size. @@ -93,10 +97,13 @@ However, we currently limit the length at 2 ^ 32 elements. In practice, even if the vector elements are single bytes, it'll allow to store more than 4 GB data in blockchain storage. +[pallet-revive-limits]: https://docs.polkadot.com/polkadot-protocol/smart-contract-basics/evm-vs-polkavm/#current-memory-limits +[static-buffer]: https://github.com/use-ink/ink/blob/master/ARCHITECTURE.md#communication-with-the-pallet + ### Caveats Iterators are not provided. `StorageVec` is expected to be used to -store a lot elements, where iterating through the elements would be +store a lot of elements, where iterating through the elements would be rather inefficient. Manually iterating over the elements using a loop is possible but considered an anti-pattern for most cases. @@ -118,7 +125,7 @@ storage read. ### Storage Layout -At given `StorageKey` `K`, the length of the `StorageVec` is hold. +At given `StorageKey` `K`, the length of the `StorageVec` is held. Each element `E` is then stored under a combination of the `StorageVec` key `K` and the elements index. diff --git a/sidebars.js b/sidebars.js index 5a89c8c0fa..2cba8e3b09 100644 --- a/sidebars.js +++ b/sidebars.js @@ -43,8 +43,8 @@ module.exports = { label: "Data Structures", items: [ "advanced/datastructures/overview", - "advanced/datastructures/storagevec", "advanced/datastructures/mapping", + "advanced/datastructures/storagevec", "advanced/datastructures/storage-layout", "advanced/datastructures/custom", "advanced/datastructures/storage-in-metadata" diff --git a/src/data/projects.tsx b/src/data/projects.tsx index 8ef8d128ac..6edfe3375f 100644 --- a/src/data/projects.tsx +++ b/src/data/projects.tsx @@ -67,7 +67,7 @@ export const projects: Project[] = [ { link: 'https://analyze.ink/', logo: 'img/projects/ink-analyzer-logo.svg', - title: 'A collection of modular and reusable libraries and tools for improving of ink! language support in IDEs/editors.', + title: 'A collection of modular and reusable libraries and tools for improving ink! language support in IDEs/editors.', about: ( <>