HDDS-9700. Allow ozone admin container info to list multiple containers

[sodonnel]

What changes were proposed in this pull request?

Currently the command:

ozone admin container info 1234

Only accepts a single container ID. If you need to list the details from many containers, then the JVM startup overhead makes the command very slow.

This Jira changes the behavior so you can pass multiple containers at the same time, eg:

ozone admin container info 1 2 456 789

Additionally, you can pass the container list over stdin, with 1 containerID per line. To do that, you pass the container id as a dash character:

ozone admin container info - < containerList

When passing multiple containers, if a container is not found or is an invalid ID, and error is printed to stdout and the command will continue with the next ID in the list.

If containers are being read from stdin, or multiple IDs are specified on the command list, the JSON output will be an array of objects rather than a series of standalone objects. For a single ID passed, the json output is not an array as it was before to ensure compatibility.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-9700

How was this patch tested?

New and changed unit tests. Also validated output in a docker compose cluster. Sample output:

bash-4.2$ ozone admin container info
Missing required parameter: '<container ID>'
Usage: ozone admin container info [-hV] [--json] [-id=<scmServiceId>]
                                  [--scm=<scm>] <container ID>...
Show information about a specific container
      <container ID>...   One or more container IDs separated by spaces. To
                            read from stdin, specify '-' and supply the
                            container IDs separated by newlines.
  -h, --help              Show this help message and exit.
      -id, --service-id=<scmServiceId>
                          ServiceId of SCM HA Cluster
      --json              Format output as JSON
      --scm=<scm>         The destination scm (host:port)
  -V, --version           Print version information and exit.


bash-4.2$ ozone admin container info 1
Container id: 1
Pipeline id: 5fdac11e-e035-44d9-bc3c-16711c2b30aa
Container State: CLOSED
Datanodes: [f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: f3171888-dbf9-4073-a6f2-86d552633f27; Location: f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]


bash-4.2$ ozone admin container info 1 2 invalid 5454 3
Container id: 1
Pipeline id: 3350506c-347d-4ff0-93d1-9543dffca5fa
Container State: CLOSED
Datanodes: [f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: f3171888-dbf9-4073-a6f2-86d552633f27; Location: f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]

Container id: 2
Pipeline id: 9068546d-7977-417b-8bcf-380b9c2d9a9c
Container State: CLOSED
Datanodes: [f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: f3171888-dbf9-4073-a6f2-86d552633f27; Location: f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]
Invalid container ID: invalid
Unable to retrieve the container details for 5454

Container id: 3
Pipeline id: d987fe3a-fbac-4075-944b-68c16845de20
Container State: CLOSED
Datanodes: [f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: f3171888-dbf9-4073-a6f2-86d552633f27; Location: f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]


bash-4.2$ ozone admin container info 1 2 invalid 5454 3 2> errs
Container id: 1
Pipeline id: 2da627a2-ae28-43dc-acb1-bf290d24d5ce
Container State: CLOSED
Datanodes: [f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: f3171888-dbf9-4073-a6f2-86d552633f27; Location: f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]

Container id: 2
Pipeline id: d8c5fd1b-409c-4733-826f-bc524ed77405
Container State: CLOSED
Datanodes: [f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: f3171888-dbf9-4073-a6f2-86d552633f27; Location: f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]

Container id: 3
Pipeline id: a88f2992-f5ef-455a-a4ab-b5bb5ab48126
Container State: CLOSED
Datanodes: [f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: f3171888-dbf9-4073-a6f2-86d552633f27; Location: f3171888-dbf9-4073-a6f2-86d552633f27/ozone_datanode_1.ozone_default]

bash-4.2$ more errs 
Invalid container ID: invalid
Unable to retrieve the container details for 5454

Reading from stdin:

bash-4.2$ cat containers | ozone admin container info -
Container id: 1
Pipeline id: fb272e11-e770-48d7-bae8-d0b0c468fad3
Container State: CLOSED
Datanodes: [2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: 2c6f320c-ddb9-4f72-9157-05ee60dc6384; Location: 2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]

Container id: 2
Pipeline id: 968dc125-3ade-4114-a640-50ac8239f2b3
Container State: CLOSED
Datanodes: [2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: 2c6f320c-ddb9-4f72-9157-05ee60dc6384; Location: 2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Invalid container ID: invaid

Container id: 3
Pipeline id: a63dc520-a5ef-4e56-b120-175530ad47e3
Container State: CLOSED
Datanodes: [2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: 2c6f320c-ddb9-4f72-9157-05ee60dc6384; Location: 2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]

Container id: 4
Pipeline id: 70f14c17-0969-47c7-a39d-40b3dd28396e
Container State: OPEN
Datanodes: [2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Replicas: [State: OPEN; ReplicaIndex: 0; Origin: 2c6f320c-ddb9-4f72-9157-05ee60dc6384; Location: 2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]


bash-4.2$ ozone admin container info - < containers 
Container id: 1
Pipeline id: 6d412aea-bb85-4ec7-81e0-ffeec2c98228
Container State: CLOSED
Datanodes: [2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: 2c6f320c-ddb9-4f72-9157-05ee60dc6384; Location: 2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]

Container id: 2
Pipeline id: 61ad0279-70bc-4460-ae2e-0bc5a7a82313
Container State: CLOSED
Datanodes: [2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: 2c6f320c-ddb9-4f72-9157-05ee60dc6384; Location: 2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Invalid container ID: invaid

Container id: 3
Pipeline id: 5ea9a9db-3263-42f4-aa9b-c77b842c2c58
Container State: CLOSED
Datanodes: [2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Replicas: [State: CLOSED; ReplicaIndex: 0; Origin: 2c6f320c-ddb9-4f72-9157-05ee60dc6384; Location: 2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]

Container id: 4
Pipeline id: 70f14c17-0969-47c7-a39d-40b3dd28396e
Container State: OPEN
Datanodes: [2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]
Replicas: [State: OPEN; ReplicaIndex: 0; Origin: 2c6f320c-ddb9-4f72-9157-05ee60dc6384; Location: 2c6f320c-ddb9-4f72-9157-05ee60dc6384/ozone_datanode_1.ozone_default]

bash-4.2$ more containers 
1
2
invaid
3
4

Similar output for the --json switch.

[adoroszlai]

Thanks @sodonnel for the improvement and the detailed PR description.