HDDS-13305. Create wrapper object for container checksums #8789
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package org.apache.hadoop.hdds.scm.container; | ||
|
|
||
| import java.util.Objects; | ||
|
|
||
| /** | ||
| * Wrapper for container checksums (data, metadata, etc.). | ||
| * Provides equality, hash, and hex string rendering. | ||
| */ | ||
| public class ContainerChecksums { | ||
|
echonesis marked this conversation as resolved.
Outdated
|
||
| private final long dataChecksum; | ||
| private final Long metadataChecksum; // nullable for future use | ||
|
Contributor
There was a problem hiding this comment. See the recently merged #8565 for how we are planning to handle unset checksums. We can add this handling into the |
||
|
|
||
| public ContainerChecksums(long dataChecksum) { | ||
| this(dataChecksum, null); | ||
| } | ||
|
|
||
| public ContainerChecksums(long dataChecksum, Long metadataChecksum) { | ||
| this.dataChecksum = dataChecksum; | ||
| this.metadataChecksum = metadataChecksum; | ||
| } | ||
|
Contributor
There was a problem hiding this comment. We could add factory methods (and make constructor private) to reduce accidental mismatch of intended and actual source (data or metadata). Something like: public static ContainerChecksums unknown(); // use constant value
public static ContainerChecksums dataOnly(long dataChecksum);
public static ContainerChecksums metadataOnly(long metadataChecksum);
public static ContainerChecksums of(long dataChecksum, long metadataChecksum); |
||
|
|
||
| public long getDataChecksum() { | ||
| return dataChecksum; | ||
| } | ||
|
|
||
| public Long getMetadataChecksum() { | ||
| return metadataChecksum; | ||
| } | ||
|
|
||
| @Override | ||
| public boolean equals(Object obj) { | ||
| if (this == obj) { | ||
| return true; | ||
| } | ||
| if (!(obj instanceof ContainerChecksums)) { | ||
| return false; | ||
| } | ||
| ContainerChecksums that = (ContainerChecksums) obj; | ||
| return dataChecksum == that.dataChecksum && | ||
| Objects.equals(metadataChecksum, that.metadataChecksum); | ||
| } | ||
|
|
||
| @Override | ||
| public int hashCode() { | ||
| return Objects.hash(dataChecksum, metadataChecksum); | ||
| } | ||
|
|
||
| @Override | ||
| public String toString() { | ||
| StringBuilder sb = new StringBuilder(); | ||
| sb.append("data=").append(Long.toHexString(dataChecksum)); | ||
| if (metadataChecksum != null) { | ||
| sb.append(", metadata=").append(Long.toHexString(metadataChecksum)); | ||
| } | ||
|
Contributor
There was a problem hiding this comment.
Contributor
Author
There was a problem hiding this comment. Got it. I will set the default metadataChecksum value to be |
||
| return sb.toString(); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package org.apache.hadoop.hdds.scm.container; | ||
|
|
||
| import static org.junit.jupiter.api.Assertions.assertEquals; | ||
| import static org.junit.jupiter.api.Assertions.assertFalse; | ||
| import static org.junit.jupiter.api.Assertions.assertNotEquals; | ||
| import static org.junit.jupiter.api.Assertions.assertTrue; | ||
|
|
||
| import org.junit.jupiter.api.Test; | ||
|
|
||
| class ContainerChecksumsTest { | ||
|
echonesis marked this conversation as resolved.
Outdated
|
||
| @Test | ||
| void testEqualsAndHashCode() { | ||
| ContainerChecksums c1 = new ContainerChecksums(123L); | ||
| ContainerChecksums c2 = new ContainerChecksums(123L); | ||
| ContainerChecksums c3 = new ContainerChecksums(456L); | ||
| ContainerChecksums c4 = new ContainerChecksums(123L, 789L); | ||
| ContainerChecksums c5 = new ContainerChecksums(123L, 789L); | ||
| ContainerChecksums c6 = new ContainerChecksums(123L, 790L); | ||
|
|
||
| assertEquals(c1, c2); | ||
| assertEquals(c1.hashCode(), c2.hashCode()); | ||
| assertNotEquals(c1, c3); | ||
| assertNotEquals(c1, c4); | ||
| assertEquals(c4, c5); | ||
| assertNotEquals(c4, c6); | ||
| } | ||
|
|
||
| @Test | ||
| void testToString() { | ||
| ContainerChecksums c1 = new ContainerChecksums(0x1234ABCDL); | ||
| assertTrue(c1.toString().contains("data=1234abcd")); | ||
| assertFalse(c1.toString().contains("metadata=")); | ||
|
echonesis marked this conversation as resolved.
Outdated
|
||
|
|
||
| ContainerChecksums c2 = new ContainerChecksums(0x1234ABCDL, 0xDEADBEEFL); | ||
| assertTrue(c2.toString().contains("data=1234abcd")); | ||
| assertTrue(c2.toString().contains("metadata=deadbeef")); | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -85,7 +85,7 @@ void testCorruptionDetected(TestContainerCorruptions corruption) | |
| assertTrue(containerChecksumFileExists(containerID)); | ||
|
|
||
| waitForScmToSeeReplicaState(containerID, CLOSED); | ||
| long initialReportedDataChecksum = getContainerReplica(containerID).getChecksums().getDataChecksum(); | ||
|
Contributor
There was a problem hiding this comment. For this use case we only need to use the There is a new
Contributor
Author
There was a problem hiding this comment. Got it. |
||
|
|
||
| corruption.applyTo(container); | ||
|
|
||
|
|
@@ -98,7 +98,7 @@ void testCorruptionDetected(TestContainerCorruptions corruption) | |
|
|
||
| // Wait for SCM to get a report of the unhealthy replica with a different checksum than before. | ||
| waitForScmToSeeReplicaState(containerID, UNHEALTHY); | ||
| long newReportedDataChecksum = getContainerReplica(containerID).getChecksums().getDataChecksum(); | ||
| if (corruption == TestContainerCorruptions.MISSING_METADATA_DIR || | ||
| corruption == TestContainerCorruptions.MISSING_CONTAINER_DIR) { | ||
| // In these cases, the new tree will not be able to be written since it exists in the metadata directory. | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need this class on the datanode too, so it needs to be in a more general package.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it.
I will move it into the path under
hadoop-hdds/common.