Skip to content

HDDS-12378. Change default hdds.scm.safemode.min.datanode to 3 #8331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nandakumar131 merged 7 commits into from
May 21, 2025
Merged

HDDS-12378. Change default hdds.scm.safemode.min.datanode to 3 #8331

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
96d71a8
Change default hdds.scm.safemode.min.datanode to 3
kostacie Apr 24, 2025
d23c395
Add warn log, fix doc
kostacie Apr 25, 2025
f4b3218
Fix doc
kostacie Apr 25, 2025
0be4df6
Fix tests and config
kostacie Apr 28, 2025
f683f37
Delete warn message, revert changes
kostacie May 21, 2025
3efcf26
Clean up
kostacie May 21, 2025
e53e08d
Delete unused test, revert changes
kostacie May 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/HddsConfigKeys.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ public final class HddsConfigKeys {
public static final boolean HDDS_SCM_SAFEMODE_ENABLED_DEFAULT = true;
public static final String HDDS_SCM_SAFEMODE_MIN_DATANODE =
"hdds.scm.safemode.min.datanode";
public static final int HDDS_SCM_SAFEMODE_MIN_DATANODE_DEFAULT = 1;
public static final int HDDS_SCM_SAFEMODE_MIN_DATANODE_DEFAULT = 3;

public static final String
HDDS_SCM_WAIT_TIME_AFTER_SAFE_MODE_EXIT =
Expand Down
2 changes: 1 addition & 1 deletion hadoop-hdds/common/src/main/resources/ozone-default.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1637,7 +1637,7 @@

<property>
<name>hdds.scm.safemode.min.datanode</name>
<value>1</value>
<value>3</value>
<tag>HDDS,SCM,OPERATION</tag>
<description>Minimum DataNodes which should be registered to get SCM out of
safe mode.
Expand Down
4 changes: 2 additions & 2 deletions hadoop-hdds/docs/content/concept/StorageContainerManager.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,6 @@ key | default | description
----|---------|------------
ozone.scm.container.size | 5GB | Default container size used by Ozone
ozone.scm.block.size | 256MB | The default size of a data block.
hdds.scm.safemode.min.datanode | 1 | Minimum number of datanodes to start the real work.
hdds.scm.safemode.min.datanode | 3 | Minimum number of datanodes to start the real work.
Copy link
Contributor

@myskov myskov Apr 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's also a Chinese version of the doc. Could you please make changes there too?

Copy link
Contributor Author

@kostacie kostacie Apr 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sure.

ozone.scm.http-address | 0.0.0.0:9876 | HTTP address of the SCM server
ozone.metadata.dirs | none | Directory to store persisted data (RocksDB).
ozone.metadata.dirs | none | Directory to store persisted data (RocksDB).
Copy link
Contributor

@myskov myskov Apr 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please revert irrelevant change

Copy link
Contributor Author

@kostacie kostacie Apr 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason I can't revert that as it changes itself and adds an extra line.

2 changes: 1 addition & 1 deletion hadoop-hdds/docs/content/concept/StorageContainerManager.zh.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,6 @@ SCM 负责创建 Ozone 集群。当通过 `init` 命令启动 SCM 时，SCM 将
----|---------|------------
ozone.scm.container.size | 5GB | Ozone 使用的默认容器的大小
ozone.scm.block.size | 256MB | 数据块的默认大小
hdds.scm.safemode.min.datanode | 1 | 能够启动实际工作所需的最小数据节点数
hdds.scm.safemode.min.datanode | 3 | 能够启动实际工作所需的最小数据节点数
ozone.scm.http-address | 0.0.0.0:9876 | SCM 服务端使用的 HTTP 地址
ozone.metadata.dirs | none | 存储持久化数据的目录（RocksDB）
1 change: 1 addition & 0 deletions ...cm/src/test/java/org/apache/hadoop/hdds/scm/safemode/TestHealthyPipelineSafeModeRule.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ public void testHealthyPipelineSafeModeRuleWithNoPipelines()
ContainerManager containerManager = mock(ContainerManager.class);
when(containerManager.getContainers()).thenReturn(containers);
config.set(HddsConfigKeys.OZONE_METADATA_DIRS, tempFile.getPath());
config.setInt(HddsConfigKeys.HDDS_SCM_SAFEMODE_MIN_DATANODE, 0);
// enable pipeline check
config.setBoolean(
HddsConfigKeys.HDDS_SCM_SAFEMODE_PIPELINE_CREATION, false);
Expand Down
1 change: 1 addition & 0 deletions .../server-scm/src/test/java/org/apache/hadoop/hdds/scm/safemode/TestSCMSafeModeManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ public void setUp() throws IOException {
config.setBoolean(HddsConfigKeys.HDDS_SCM_SAFEMODE_PIPELINE_CREATION,
false);
config.set(HddsConfigKeys.OZONE_METADATA_DIRS, tempDir.getAbsolutePath());
config.setInt(HddsConfigKeys.HDDS_SCM_SAFEMODE_MIN_DATANODE, 1);
scmMetadataStore = new SCMMetadataStoreImpl(config);
}

Expand Down
1 change: 1 addition & 0 deletions hadoop-ozone/dist/src/main/compose/ozonescripts/docker-config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ OZONE-SITE.XML_ozone.scm.block.client.address=scm
OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
OZONE-SITE.XML_ozone.scm.client.address=scm
OZONE-SITE.XML_ozone.server.default.replication=1
OZONE-SITE.XML_hdds.scm.safemode.min.datanode=1
OZONE-SITE.XML_hdds.datanode.dir=/data/hdds
OZONE-SITE.XML_hdds.datanode.volume.min.free.space=100MB
OZONE-SITE.XML_hdds.scmclient.max.retry.timeout=30s
Expand Down