HDDS-3185 Construct a standalone ratis server for SCM.

[timmylicheng]

What changes were proposed in this pull request?

Standalone SCM RatisServer

(Please fill in changes proposed in this fix)

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-3185

(Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HDDS-XXXX. Fix a typo in YYY.)

Please replace this section with the link to the Apache JIRA)

How was this patch tested?

UT
(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)

[elek]

Thanks to work on this @timmylicheng

I am confused a little bit because this was the last message on the mailing list:

I've had several sync-up calls offline about SCM HA's design principles and basic ideas.

The doc in the link is in draft and due to the complexity of this feature, I'm still trying my best best to add more details into this doc.

Will keep you guys posted on the progress. And I will schedule a review meeting with Ozone community on this in the following weeks.

https://lists.apache.org/thread.html/r68b9de5a0cb1adda449aabb9391e9baed1e06e74e0107f57cd034811%40%3Cozone-dev.hadoop.apache.org%3E

It's just not clear for me if the design is final and the implementation is started or still under discussion?

[timmylicheng]

@elek Hey Marton,
Sorry this patch was meant to merge into HDDS-2823 as a dev branch for SCM HA.

So far the doc work is ongoing in parallel with prototyping.
The doc still needs improvements and I will find time to work on that.
This patch is only to work on a prototype of SCM HA to get a feeling of it. To have Ratis server and state machine for SCM doesn't have debates. Current discussion for design is to collect how to handle Raft transaction for all different types of actions and reports on SCM. Besides going thru the current code base to make analysis, I'm also probing by prototyping. I start off implementing a stand alone Ratis Server and make steps.

I will try to finalize the design doc and schedule a call with the community to include more guys.

[ChenSammi]

+1

[elek]

@timmylicheng Thanks the explain it.

I will add your explanation to the next Community Meeting minutes.

SCM-HA: First draft is already available and will be updated soon based on the existing feedback. Prototype implementation has been started at HDDS-2823 branch. 

[elek]

Speaking about the code in the patch: I would suggest to use the new config based annotation model for new code:

https://cwiki.apache.org/confluence/display/HADOOP/Java-based+configuration+API

(I know it's not used everywhere, therefore it's hard to notice this movement, and some features can be still missing, but new code seems to be a good opportunity to switch to the new API)

[timmylicheng]

https://cwiki.apache.org/confluence/display/HADOOP/Java-based+configuration+API

@elek This new interface actually looks nice! I have a refactor plan for OM and SCM Ratis chain. I will try to leverage with it in the refactoring work. The current way to populate configs needs some changes.

[elek]

I will try to leverage with it in the refactoring work.

Thanks a lot. I am fine if it's a long-term plan (and we may need improvements on the Configuration API side, too). This API just so hidden (yet) that am trying to make it more widely known... ;-)