Skip to content

HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. #2155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
bshashikant merged 10 commits into from
Apr 23, 2021
Merged

HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. #2155

bshashikant merged 10 commits into from
Apr 23, 2021

Conversation

@bshashikant
Copy link
Contributor

@bshashikant bshashikant commented Apr 14, 2021

What changes were proposed in this pull request?

Fixed install snapshot behaviour

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-5103

How was this patch tested?

Enabled Unit tests

bharatviswa504
bharatviswa504 reviewed Apr 14, 2021
View reviewed changes
@GlenGeng-awx
Copy link
Contributor

GlenGeng-awx commented Apr 14, 2021

Thanks @bshashikant. LGTM.
BTW, does the ratis version in master contains the fix RATIS-1326 ?

@bshashikant
Copy link
Contributor Author

bshashikant commented Apr 19, 2021

Thanks @bshashikant. LGTM.
BTW, does the ratis version in master contains the fix RATIS-1326 ?

This needs https://issues.apache.org/jira/browse/RATIS-1356 as well. Once this gets resolved, will update the ratis version.

@bshashikant bshashikant mentioned this pull request Apr 19, 2021
Merged
bharatviswa504
bharatviswa504 reviewed Apr 20, 2021
View reviewed changes
Copy link
Contributor

@bharatviswa504 bharatviswa504 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM.
Few minor comments. Now I see leader address in logs.

GlenGeng-awx
GlenGeng-awx reviewed Apr 20, 2021
View reviewed changes
bharatviswa504
bharatviswa504 approved these changes Apr 20, 2021
View reviewed changes
Copy link
Contributor

@bharatviswa504 bharatviswa504 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 LGTM

@bshashikant bshashikant mentioned this pull request Apr 20, 2021
Merged
@bshashikant
Copy link
Contributor Author

bshashikant commented Apr 23, 2021

Thanks @bharatviswa504 / @GlenGeng for the review. Committing this.

@bshashikant bshashikant merged commit bef180e into apache:master Apr 23, 2021
@elek
Copy link
Member

elek commented Apr 26, 2021

Hi, this patch modified Ozone to use a -SNAPSHOT dependency. Can we avoid using it? Is it absolutely necessary?

I would prefer to depend on released Ratis artifacts only as our last release was delayed with 4 months when we were blocked by the Ratis release.

Can we have a ratis 2.0.1 and depend on it?

@bshashikant
Copy link
Contributor Author

bshashikant commented Apr 26, 2021

Hi, this patch modified Ozone to use a -SNAPSHOT dependency. Can we avoid using it? Is it absolutely necessary?

I would prefer to depend on released Ratis artifacts only as our last release was delayed with 4 months when we were blocked by the Ratis release.

Can we have a ratis 2.0.1 and depend on it?

This is necessary as only the snapshot has the required fixes to make snapshot mechanism work for SCM HA.

@elek
Copy link
Member

elek commented Apr 26, 2021

This is necessary as only the snapshot has the required fixes to make snapshot mechanism work for SCM HA.

Thanks for the answer. Still I would prefer to depend only on released artifacts from other projects (which is suggested originally by Nanda) I would ask for other's opinion on the community meeting.

Can you please help me with the required commit from Ratis side? Is it apache/ratis#460? If it's just one commit I will start a ratis discussion to release 2.0.1, and we can switch back to a released version...

errose28 added a commit to errose28/ozone that referenced this pull request May 4, 2021
@errose28
Merge remote-tracking branch 'upstream/master' into HDDS-3698-nonroll…
253cc55 
…ing-upgrade-master-merge2

* upstream/master: (56 commits)
  HDDS-2212. Genconf tool should generate config files for secure clust… (apache#1788)
  HDDS-5166. Remove duplicate assignment of OZONE_OPTS for freon and sh (apache#2195)
  Revert "HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)"
  HDDS-4983. Display key offset for each block in command key info (apache#2051)
  HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)
  HDDS-4585. Support bucket acl operation in S3g (apache#1701)
  HDDS-5153. Decommissioning a dead node should complete immediately (apache#2190)
  HDDS-5147. Intermittent test failure in TestContainerDeletionChoosingPolicy#testRandomChoosingPolicy (apache#2188)
  HDDS-5152. Fix Suggested leader in Client. (apache#2189)
  HDDS-5148. Bump ratis version to 2.1.0-ff8aa66-SNAPSHOT (apache#2184)
  HDDS-4515. Datanodes should be able to persist and load CRL (apache#2181)
  HDDS-5060. [SCM HA Security] Make InterSCM grpc channel secure. (apache#2187)
  HDDS-5051. Ensure failover to suggested leader if any for NotLeaderException. (apache#2141)
  HDDS-5127. Fix getServiceList when SCM HA is enabled (apache#2173)
  HDDS-4889. Add simple CI check for docs (apache#2156)
  HDDS-5131. Use timeout in github actions (apache#2176)
  HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. (apache#2155)
  HDDS-5124. Use OzoneConsts.OZONE_TIME_ZONE instead of "GMT" (apache#2166)
  HDDS-5047. Refactor Pipeline to use ReplicationConfig instead of factor/type (apache#2096)
  HDDS-5083. Bump version of common-compress (apache#2139)
  ...

Conflicts:
	hadoop-hdds/common/pom.xml
	hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java
	hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java
	hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/HddsDatanodeService.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManager.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMStorageConfig.java
	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java
	hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java
	hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneClusterImpl.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMStorage.java
	hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java
	hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconStorageContainerManagerFacade.java
errose28 added a commit to errose28/ozone that referenced this pull request May 13, 2021
@errose28
Merge branch 'HDDS-3698-nonrolling-upgrade' into HDDS-5138-admin-chec…
080019b 
…k-in-auth

* HDDS-3698-nonrolling-upgrade: (57 commits)
  Fix compilation errors afte merge Update javassist in recon pom Fix changes introduced in merge that failed TestSCMNodeManager upgrade tests Fix checkstyle Fix intermittent test failure TestSCMNodeManager#testSetNodeOpStateAndCommandFired after merge Skip scm init default layout version in TestOzoneConfigurationFields
  HDDS-2212. Genconf tool should generate config files for secure clust… (apache#1788)
  HDDS-5166. Remove duplicate assignment of OZONE_OPTS for freon and sh (apache#2195)
  Revert "HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)"
  HDDS-4983. Display key offset for each block in command key info (apache#2051)
  HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)
  HDDS-4585. Support bucket acl operation in S3g (apache#1701)
  HDDS-5153. Decommissioning a dead node should complete immediately (apache#2190)
  HDDS-5147. Intermittent test failure in TestContainerDeletionChoosingPolicy#testRandomChoosingPolicy (apache#2188)
  HDDS-5152. Fix Suggested leader in Client. (apache#2189)
  HDDS-5148. Bump ratis version to 2.1.0-ff8aa66-SNAPSHOT (apache#2184)
  HDDS-4515. Datanodes should be able to persist and load CRL (apache#2181)
  HDDS-5060. [SCM HA Security] Make InterSCM grpc channel secure. (apache#2187)
  HDDS-5051. Ensure failover to suggested leader if any for NotLeaderException. (apache#2141)
  HDDS-5127. Fix getServiceList when SCM HA is enabled (apache#2173)
  HDDS-4889. Add simple CI check for docs (apache#2156)
  HDDS-5131. Use timeout in github actions (apache#2176)
  HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. (apache#2155)
  HDDS-5124. Use OzoneConsts.OZONE_TIME_ZONE instead of "GMT" (apache#2166)
  HDDS-5047. Refactor Pipeline to use ReplicationConfig instead of factor/type (apache#2096)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bharatviswa504 bharatviswa504 bharatviswa504 approved these changes

+1 more reviewer

@GlenGeng-awx GlenGeng-awx GlenGeng-awx left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@bshashikant @GlenGeng-awx @elek @bharatviswa504