HDDS-7817. Add HTTP and HTTPS ports to DatanodeDetails

[dombizita]

What changes were proposed in this pull request?

previous this change the DatanodeDetails didn't store any information regarding the HTTP and HTTPS ports of the datanode. there was a conversation on how to add the ports in a backward compatible way and we decided to add a new HDDS layout version as WEBUI_PORTS_IN_DATANODEDETAILS. the original task got separated into two more tasks that was done prior this. one to handle the the 1.4.0 upgrade test, which got covered by #4456 and one to add a layout version for the RATIS_DATASTREAM port, which was done in #4500.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-7817

How was this patch tested?

green CI on my fork: https://github.com/dombizita/ozone/actions/runs/4369081456/jobs/7650370829

[dombizita]

please take a look at it @fapifta @adoroszlai @errose28

[fapifta]

Thank you @dombizita for working on this piece.

I have one minor comment inline.
Besides that I think it would be useful to create a test which checks the case where the metadata layout version in the DatanodeLayoutStorage managed VERSION file is still under 5, and check if the createDatanodeIdFile call writes just the expected ports, while if it is already 5, then it writes the HTTP/HTTPS ports as well, would you mind add these tests?

[adoroszlai]

Thanks @dombizita for working on this.

I agree with @fapifta about the need for adding a unit test (in TestDatanodeIdYaml).

(Note: please update from your fork, I've merged master into your branch and resolved some conflicts caused by my latest commit.)

[fapifta]

I have restarted the failed test as it seemed an irrelevant failure (SCM stuck in safe mode after successfully exiting a couple of times already).

I am +1 for the changes to be committed, but I would like to see if @errose28 can also review, and how @adoroszlai sees my opinion regarding the RATIS_DATASTREAM port.