Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 78 additions & 0 deletions hadoop-hdds/docs/content/feature/OM-HA.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,84 @@ ozone om [global options (optional)] --bootstrap --force

Note that using the _force_ option during bootstrap could crash the OM process if it does not have updated configurations.

## OM Leader Transfer

The `ozone admin om transfer` command allows you to manually transfer the leadership of the Ozone Manager (OM) Raft group to a specific OM node or to a randomly chosen follower.

### Usage

```bash
ozone admin om transfer -id <OM_SERVICE_ID> -n <NEW_LEADER_ID>
ozone admin om transfer -id <OM_SERVICE_ID> -r
```

* `-id, --service-id`: Specifies the Ozone Manager Service ID.
* `-n, --newLeaderId, --new-leader-id`: The node ID of the OM to which leadership will be transferred (e.g., `om1`).
* `-r, --random`: Randomly chooses a follower to transfer leadership to.

### Example

To transfer leadership to `om2` in a cluster with service ID `cluster1`:

```bash
ozone admin om transfer -id cluster1 -n om2
```

To transfer leadership to a random follower:

```bash
ozone admin om transfer -id cluster1 -r
```

## OM Service Roles Listing

The `ozone admin om roles` command lists all Ozone Managers and their respective Raft server roles (leader, follower, or candidate).

### Usage

```bash
ozone admin om roles [-id <OM_SERVICE_ID>] [--json | --table]
```

* `-id, --service-id`: (Optional) Specifies the Ozone Manager Service ID.
* `--json`: (Optional) Formats the output as JSON.
* `--table`: (Optional) Formats the output as a table.

### Example

To list OM roles for `cluster1`:

```bash
ozone admin om roles -id cluster1
```

Example output:

```
om1 : LEADER (host1)
om2 : FOLLOWER (host2)
om3 : FOLLOWER (host3)
```

To list OM roles as a table:

```bash
ozone admin om roles -id cluster1 --table
```

Example table output:

```
Ozone Manager Roles
-------------------
Host Name | Node ID | Role
-------------------
host1 | om1 | LEADER
host2 | om2 | FOLLOWER
host3 | om3 | FOLLOWER
-------------------
```

## Automatic Snapshot Installation for Stale Ozone Managers

Sometimes an OM follower node may be offline or fall far behind the OM leader's raft log.
Expand Down
77 changes: 77 additions & 0 deletions hadoop-hdds/docs/content/feature/SCM-HA.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,83 @@ ozone scm --bootstrap

Note: both commands perform one-time initialization. SCM still needs to be started by running `ozone --daemon start scm`.

## SCM Leader Transfer

The `ozone admin scm transfer` command allows you to manually transfer the leadership of the Storage Container Manager (SCM) Raft group to a specific SCM node or to a randomly chosen follower.

### Usage

```bash
ozone admin scm transfer -id <SCM_SERVICE_ID> -n <NEW_LEADER_ID>
ozone admin scm transfer -id <SCM_SERVICE_ID> -r
```

* `-id, --service-id`: Specifies the SCM Service ID.
* `-n, --newLeaderId, --new-leader-id`: The node ID of the SCM to which leadership will be transferred (e.g., `scm1`).
* `-r, --random`: Randomly chooses a follower to transfer leadership to.

### Example

To transfer leadership to `scm2` in a cluster with service ID `cluster1`:

```bash
ozone admin scm transfer -id cluster1 -n scm2
```

To transfer leadership to a random follower:

```bash
ozone admin scm transfer -id cluster1 -r
```

## SCM Service Roles Listing

The `ozone admin scm roles` command lists all Storage Container Managers and their respective Raft server roles (leader, follower, or candidate).

### Usage

```bash
ozone admin scm roles [--json | --table]
```

* `--json`: (Optional) Formats the output as JSON.
* `--table`: (Optional) Formats the output as a table.

### Example

To list SCM roles:

```bash
ozone admin scm roles
```

Example output:

```
host1:9876:LEADER:scm1:192.168.1.1
host2:9876:FOLLOWER:scm2:192.168.1.2
host3:9876:FOLLOWER:scm3:192.168.1.3
```

To list SCM roles as a table:

```bash
ozone admin scm roles --table
```

Example table output:

```
Storage Container Manager Roles
---------------------------------------------------
Host Name | Ratis Port | Role | Node ID | Host Address
---------------------------------------------------
host1 | 9876 | LEADER | scm1 | 192.168.1.1
host2 | 9876 | FOLLOWER | scm2 | 192.168.1.2
host3 | 9876 | FOLLOWER | scm3 | 192.168.1.3
---------------------------------------------------
```

## Auto-bootstrap

In some environments (e.g. Kubernetes) we need to have a common, unified way to initialize SCM HA quorum. As a reminder, the standard initialization flow is the following:
Expand Down