Hbase-24764 : Add support of adding base peer configs via hbase-site.xml for all replication peers #2327
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ml for all replication peers
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Contributor
Author
|
@bharathv : Can you please have a look at this ? This is for branch-1. |
Contributor
|
Will do, waiting for a +1 from Stack on the original PR and then I can sign off on this. |
Contributor
Author
|
@bharathv : Can you take a look at this. Thanks. |
bharathv
approved these changes
Sep 16, 2020
Contributor
bharathv
left a comment
bharathv
left a comment
There was a problem hiding this comment.
Add a quick note in the commit message about how the replication design changes from branch-1 vs other branches affect this? (The part about multiple RS updating the ZK state and the last RPC wins and if there is a divergence in configuration across RS, the end result may be non-deterministic).
| String configName = entry.getKey(); | ||
| String configValue = entry.getValue(); | ||
| // Only override if base config does not exist in existing replication peer configs | ||
| if (!receivedPeerConfigMap.containsKey(configName)) { |
Contributor
There was a problem hiding this comment.
nit: just use this.configuration directly?
…t to other branch for Hbase-24764. branch-1 does not have master based coordination for replication like master branch , and hence each RS is responsible for initing peers and updating ZK states. As part of this change we are updating zk state of peers after reading from configuration, so if there is a divergence in configuration across RS the result can be can be non-deteministic and the last RS RPC will win.
|
🎊 +1 overall
This message was automatically generated. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
JIRA: https://issues.apache.org/jira/browse/HBASE-24764
JIRA Description:
Today, if a user needs to apply some common base peer configs to all the replication peers on the cluster, the only way is to execute update_peer_config via CLI which requires manual intervention and can be tedious in case of large deployment fleet.
As part of this JIRA, we plan to add the support to have base replication peer configs as part of hbase-site.xml like hbase.replication.peer.base.config="k1=v1;k2=v2.." which can be easily updated and applied as part of a rolling restart.
Example below:
This property will be empty by default, but user can override to have base configs in place.
The final peer configuration would be a merge of whatever is currently present or what users override during the peer creation/update (if any) + this newly added base config.
Related Jira: https://issues.apache.org/jira/browse/HBASE-17543. HBASE-17543 added the support to add the WALEntryFilters to default endpoint via peer configuration.
By this new Jira we are extending the support to update peer configs via hbase-site.xml and hence WalEntryFilters or any other peer property can be applied just by rolling restart.
master branch PR : #2284