storage: implement substore support#3131

Merged

erwanor merged 77 commits into

mainfrom

storage-substore

Nov 15, 2023

Commits on Nov 3, 2023

wip: hack on substore support

hdevalence
authored and
Erwan
committed
storage: `pub(crate)` *store
Erwan
committed
storage: create `multistore::Multistore`
Erwan
committed
storage(substore): make substore prefix public
Erwan
committed
storage(snapshot): dispatch reads to substores in `get_raw`
Erwan
committed
storage: basic commit strategy TODO
Erwan
committed
storage(substore): return an iterator over the config
Erwan
committed
storage(lib): `fmt`
Erwan
committed
storage: explicit initialization via `Storage::init`
Erwan
committed
storage: equip `Snapshot` with a multistore
Erwan
committed
storage(substore): add note about `rocksdb::OptimisticTransactionDB`
Erwan
committed
storage: turn the transparent store into a substore
Erwan
committed
storage(snapshot): route reads to `SubstoreSnapshot`
Erwan
committed
storage(snapshot): default to global store for tests
Erwan
committed
storage(snapshot): consolidate `Snapshot` api
Erwan
committed
storage(snapshot): implement substore dispatch for prefix queries
Erwan
committed
storage(snapshot): implement dispatch for `StateRead`
Erwan
committed
storage(snapshot): implement dispatch for range queries
Erwan
committed
storage: simplify multistore api
Erwan
committed
penumbra(storage): root store has an empty prefix
Erwan
committed
storage(substore): move all tree accesses to `SubstoreSnapshot`
Erwan
committed
penumbra(storage): update the hardcoded cf handles (tmp)
Erwan
committed
penumbra(storage): subsume db handles into `SubstoreSnapshot`
Erwan
committed
penumbra(storage): use `cf_*` naming
Erwan
committed
penumbra(storage): better error message in cf getters
Erwan
committed
storage(substore): add `SubstoreSnapshot::root_hash`
Erwan
committed
storage(substore): add `SubstoreSnapshot::get_with_proof`
Erwan
committed
storage(snapshot): forward proof calls to substore impl
Erwan
committed
storage(snapshot): fetch substore root hashes in a blocking task
Erwan
committed
storage(snapshot): ics23 double proof scaffolding
Erwan
committed
storage(snapshot): empty prefix in tmp spec
Erwan
committed
storage: implement `commit_inner_substore`
Erwan
committed
storage: shard unwritten change caches by substore
Erwan
committed
storage(substore): add derives to `SubstoreConfig`
Erwan
committed
penumbra(storage): stub commit method and rename main store
Erwan
committed
penumbra(storage): remove tree reader impl on `Snapshot`
Erwan
committed
penumbra(storage): remove stub proof gen code
Erwan
committed
penumbra(storage): implement `SubstoreStorage`
Erwan
committed
penumbra(storage): persist view in `SubstoreStorage::commit`
Erwan
committed
penumbra(storage): clean-up of TODO comments
Erwan
committed
penumbra(storage): move prefix sharding on `Cache`
Erwan
committed
penumbra(storage): remove ad-hoc `get_rightmost_leaf` method
Erwan
committed
penumbra(storage): move `VersionedKeyhash` to `store` crate
Erwan
committed
penumbra(storage): add note on `jh_dispatcher`
Erwan
committed
penumbra(storage): implement a first version of `multistore::VersionCache`
Erwan
committed
penumbra(storage): add an iterator of `MultistoreConfig`
erwanor
committed
penumbra(storage): use `new_substore_version`
erwanor
committed
penumbra(storage): default on empty cache during commit

erwanor
authored and
Erwan
committed
penumbra(storage): initialize main store version

erwanor
authored and
Erwan
committed
penumbra(storage): cleanup commit logic
Erwan
committed
storage: default on an empty changeset in `commit`
Erwan
committed
chore: run `fmt`
Erwan
committed
storage: reorganize multistore crate
Erwan
committed

Commits on Nov 7, 2023

Commits on Nov 8, 2023

storage(substore): make writes to rocksdb atomic
Erwan
committed
storage(substore): fix formatting
Erwan
committed
storage(substore): update multistore cache once commit succeeds
Erwan
committed
storage: clean-up
Erwan
committed

Commits on Nov 9, 2023

Commits on Nov 10, 2023

storage: add comments, remove obsolete TODOs
Erwan
committed

Commits on Nov 13, 2023

storage: read prefix list from config cf
Erwan
committed
storage: rename init/load
Erwan
committed
storage: use `expect` for db write
Erwan
committed

Commits on Nov 15, 2023

storage: improve comments
Erwan
committed