Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce dual-storesupport #2734

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

MathieuDutSik
Copy link
Contributor

Motivation

The dual.rs provides support for the dual stores. However, it was not implemented. This PR addresses that.

Proposal

The following was done:

  • The fn create should creates an error if the namespace already exists. This was added to the admin_test admin tests.
  • This led to some correction to the linera-storage-service, dual, memory and rocksdb backends.
  • The DualStore was introduced to the linera-client/src/storage.rs.
  • The DEFAULT_ROOT_KEY and DEFAULT_NAMESPACE are introduced in the linera-storageand used.
  • The wallet.rs is restructured. One consequence is that the initialization puts the chains in all the shards for RocksDb.

Test Plan

(A) The CI should provide the consistency check.

(B) The usage of the dualstore in the tests adds more work to the CI, so it was limited to the counter and amm tests.

(C) The runtime is not optimistic. For 19 tests the runtime is 1415 seconds with dualstoreRocksDb_Scylla and 928 seconds for running with the ScyllaDb. Therefore, the dual store is not recommended to be used. But its insertion is I think valuable to the source code.

Release Plan

The implementation of the system into TestNet / DevNet is quite premature. So, right now, it is

Links

@MathieuDutSik MathieuDutSik marked this pull request as ready for review October 29, 2024 11:33
@ma2bd
Copy link
Contributor

ma2bd commented Nov 11, 2024

@MathieuDutSik Since the DualStore is going to be used primarily for testing and benchmarking, would it make sense to repurpose this PR to provide a (hopefully) faster in-memory solution: InProcessMemory + StorageService ?

@ma2bd
Copy link
Contributor

ma2bd commented Nov 11, 2024

For 19 tests the runtime is 1415 seconds with dualstoreRocksDb_Scylla and 928 seconds for running with the ScyllaDb.

Wow, this is surprising. Do we have an explanation?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants