Skip to content

Make ScyllaDB replication factor configurable #3788

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 9, 2025

Conversation

ndr-ds
Copy link
Contributor

@ndr-ds ndr-ds commented Apr 16, 2025

Motivation

We'll use replication factor of 1 in CI and locally, but probably will use 3 in deployed networks on GCP

Proposal

Add an option to support that

Test Plan

CI + deployed network locally

Release Plan

  • Nothing to do / These changes follow the usual release cycle.

Copy link
Contributor Author

ndr-ds commented Apr 16, 2025

@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 6fd307a to 5e7a71d Compare April 16, 2025 22:42
@ndr-ds ndr-ds requested review from afck, ma2bd, MathieuDutSik and Twey April 16, 2025 23:47
@ndr-ds ndr-ds marked this pull request as ready for review April 16, 2025 23:47
@@ -1339,7 +1343,7 @@ impl ClientOptions {
let storage_config = self.storage_config()?;
debug!("Running command using storage configuration: {storage_config}");
let store_config = storage_config
.add_common_config(self.common_config())
.add_common_config(self.common_config(), self.storage_replication_factor)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it what the common config was about though?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I can add it to the common config

Copy link
Contributor

@MathieuDutSik MathieuDutSik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First comment is that the addition of a storage-replication-factor is generally a good idea. I think that in operations, the ScyllaDb instance is not spun out in that way. But, it is a nice thing to have it available for tests.

Where I think this is more problematic is where should we put this setting.

As is apparent, this is not an internal setting of ScyllaDB. The ScyllaDB instance is already running, either in Docker or some other system. Therefore, the replication factor should be next to the uri and should be obtained when parsing the --storage entry.

@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 5e7a71d to 8691581 Compare April 30, 2025 19:56
@ndr-ds ndr-ds changed the base branch from main to graphite-base/3788 May 2, 2025 19:47
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 8691581 to e6786ca Compare May 2, 2025 19:48
@ndr-ds ndr-ds changed the base branch from graphite-base/3788 to 05-02-add_local_ssd_support May 2, 2025 19:48
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from e6786ca to 11e93bc Compare May 2, 2025 20:05
@ndr-ds ndr-ds force-pushed the 05-02-add_local_ssd_support branch from 1082be6 to c0bae58 Compare May 5, 2025 21:00
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch 3 times, most recently from 6b804f1 to 4930e8e Compare May 6, 2025 02:43
@ndr-ds ndr-ds force-pushed the 05-02-add_local_ssd_support branch from c0bae58 to fa8dbb4 Compare May 6, 2025 14:51
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 4930e8e to 22d7171 Compare May 6, 2025 14:51
@ndr-ds ndr-ds force-pushed the 05-02-add_local_ssd_support branch from fa8dbb4 to dac015a Compare May 6, 2025 17:34
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 22d7171 to 32eb499 Compare May 6, 2025 17:34
@ndr-ds ndr-ds force-pushed the 05-02-add_local_ssd_support branch from dac015a to b95af69 Compare May 6, 2025 20:50
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 32eb499 to 05a929a Compare May 6, 2025 20:50
@ndr-ds ndr-ds force-pushed the 05-02-add_local_ssd_support branch from b95af69 to f5e964e Compare May 6, 2025 22:17
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 05a929a to 0d5275a Compare May 6, 2025 22:17
@ndr-ds ndr-ds force-pushed the 05-02-add_local_ssd_support branch from f5e964e to a078dbc Compare May 7, 2025 17:10
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 0d5275a to 46bf043 Compare May 7, 2025 17:10
@ndr-ds ndr-ds force-pushed the 05-02-add_local_ssd_support branch from a078dbc to 48fbe06 Compare May 7, 2025 18:23
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 46bf043 to 5f0e62a Compare May 7, 2025 18:23
@ndr-ds ndr-ds force-pushed the 05-02-add_local_ssd_support branch from 48fbe06 to c9e259d Compare May 7, 2025 19:13
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 5f0e62a to 1971bcf Compare May 7, 2025 19:13
@@ -887,6 +890,7 @@ impl TestKeyValueStore for JournalingKeyValueStore<ScyllaDbStoreInternal> {
let common_config = CommonStoreInternalConfig {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to remove this common config because most options there only work for a subset of the databases

@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 1971bcf to 2a07fcc Compare May 9, 2025 16:57
@ndr-ds ndr-ds force-pushed the 05-02-add_local_ssd_support branch from c9e259d to 1cf0afb Compare May 9, 2025 16:57
Copy link
Contributor Author

ndr-ds commented May 9, 2025

Merge activity

  • May 9, 5:25 PM EDT: A user started a stack merge that includes this pull request via Graphite.
  • May 9, 5:35 PM EDT: Graphite rebased this pull request as part of a merge.
  • May 9, 5:35 PM EDT: @ndr-ds merged this pull request with Graphite.

@ndr-ds ndr-ds changed the base branch from 05-02-add_local_ssd_support to graphite-base/3788 May 9, 2025 21:32
@ndr-ds ndr-ds changed the base branch from graphite-base/3788 to main May 9, 2025 21:33
@ndr-ds ndr-ds force-pushed the 04-14-make_scylladb_replication_factor_configurable branch from 2a07fcc to 4f383ee Compare May 9, 2025 21:34
@ndr-ds ndr-ds merged commit 2ea0e26 into main May 9, 2025
28 checks passed
@ndr-ds ndr-ds deleted the 04-14-make_scylladb_replication_factor_configurable branch May 9, 2025 21:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants