Skip to content

HDDS-8975. Clarify SCM HA auto-bootstrap doc #5021

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
adoroszlai merged 2 commits into from
Jul 5, 2023
Merged

HDDS-8975. Clarify SCM HA auto-bootstrap doc #5021

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
66b3da6
HDDS-8975. Clarify SCM HA auto-bootstrap doc
adoroszlai Jul 3, 2023
53224af
--daemon start
adoroszlai Jul 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 25 additions & 8 deletions hadoop-hdds/docs/content/feature/SCM-HA.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,28 +91,45 @@ For reliable HA support choose 3 independent nodes to form a quorum.

## Bootstrap

The initialization of the **first** SCM-HA node is the same as a none-HA SCM:
The initialization of the **first** SCM-HA node is the same as a non-HA SCM:

```
bin/ozone scm --init
ozone scm --init
```

Second and third nodes should be *bootstrapped* instead of init. These clusters will join to the configured RAFT quorum. The id of the current server is identified by DNS name or can be set explicitly by `ozone.scm.node.id`. Most of the time you don't need to set it as DNS based id detection can work well.

```
bin/ozone scm --bootstrap
ozone scm --bootstrap
```

Note: both commands perform one-time initialization. SCM still needs to be started by running `ozone scm`.

@ChenSammi ChenSammi Jul 5, 2023

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest use ozone scm --daemon start here, and line 131.

@adoroszlai adoroszlai Jul 5, 2023

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion, updated.


## Auto-bootstrap

In some environment -- such as containerized / K8s environment -- we need to have a common, unified way to initialize SCM HA quorum. As a remained, the standard initialization flow is the following:
In some environments (e.g. Kubernetes) we need to have a common, unified way to initialize SCM HA quorum. As a reminder, the standard initialization flow is the following:

1. On the first, "primordial" node: `ozone scm --init`
2. On second/third nodes: `ozone scm --bootstrap`

This can be improved: primordial SCM can be configured by setting `ozone.scm.primordial.node.id` in the config to one of the nodes.

```XML
<property>
<name>ozone.scm.primordial.node.id</name>
<value>scm1</value>
</property>
```

1. On the first, "primordial" node, call `scm --init`
2. On second/third nodes call `scm --bootstrap`
With this configuration both `scm --init` and `scm --bootstrap` can be safely executed on **all** SCM nodes. Each node will only perform the action applicable to it based on the `ozone.scm.primordial.node.id` and its own node ID.

This can be changed with using `ozone.scm.primordial.node.id`. You can define the primordial node. After setting this node, you should execute **both** `scm --init` and `scm --bootstrap` on **all** nodes.
Note: SCM still needs to be started by running `ozone scm` after the init/bootstrap process.

Based on the `ozone.scm.primordial.node.id`, the init process will be ignored on the second/third nodes and bootstrap process will be ignored on all nodes except the primordial one.
```
ozone scm --init
ozone scm --bootstrap
ozone scm
```

## SCM HA Security

Expand Down