Skip to content

Conversation

@sodonnel
Copy link
Contributor

What changes were proposed in this pull request?

Some earlier Jiras added the feature to retrieve the checksum of an Ozone key either as COMPOSITE_CRC or MD5 or the chunk checksum MD5s.

This checksum is only accessible via the Hadoop API.

This Jira adds a CLI command to Ozone to retrieve the same information.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-8133

How was this patch tested?

New unit test and manually in docker cluster:

bash-4.2$ ozone sh key checksum /vol1/bucket/512    
{
  "volumeName" : "vol1",
  "bucketName" : "bucket",
  "name" : "512",
  "dataSize" : 512,
  "algorithm" : "COMPOSITE-CRC32",
  "checksum" : "5A4FA397"
}

@sodonnel sodonnel changed the title HDDS-8133. Create ozone sh key checksum command HDDS-8133. Create ozone sh key checksum command Mar 10, 2023
@sodonnel sodonnel changed the title HDDS-8133. Create ozone sh key checksum command HDDS-8133. Create ozone sh key checksum command Mar 10, 2023
Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one minor suggestion for option name.

…/keys/ChecksumKeyHandler.java

Co-authored-by: Doroszlai, Attila <[email protected]>
@adoroszlai adoroszlai merged commit 3d036f4 into apache:master Mar 12, 2023
errose28 added a commit to errose28/ozone that referenced this pull request Mar 16, 2023
* master: (262 commits)
  HDDS-8153. Integrate ContainerBalancer with MoveManager (apache#4391)
  HDDS-8090. When getBlock from a datanode fails, retry other datanodes. (apache#4357)
  HDDS-8163 Use try-with-resources to ensure close rockdb connection in SstFilteringService (apache#4402)
  HDDS-8065. Provide GNU long options (apache#4394)
  HDDS-7930. [addendum] input stream does not refresh expired block token.
  HDDS-7930. input stream does not refresh expired block token. (apache#4378)
  HDDS-7740. [Snapshot] Implement SnapshotDeletingService (apache#4244)
  HDDS-8076. Use container cache in Key listing API. (apache#4346)
  HDDS-8091. [addendum] Generate list of config tags from ConfigTag enum - Hadoop 3.1 compatibility fix (apache#4374)
  HDDS-8144. TestDefaultCertificateClient#testTimeBeforeExpiryGracePeriod fails as we approach DST. (apache#4382)
  HDDS-8151. Support fine grained lifetime for root CA certificate (apache#4386)
  HDDS-8150. RpcClientTest and ConfigurationSourceTest not run due to naming convention (apache#4388)
  HDDS-8131. Add Configuration for OM Ratis Log Purge Tuning Parameters. (apache#4371)
  HDDS-8133. Create ozone sh key checksum command (apache#4375)
  HDDS-8142. Check if no entries in Block DB for a container on container delete (apache#4379)
  HDDS-8118. Fail container delete on non empty chunks dir (apache#4367)
  HDDS-8028. JNI for RocksDB SST Dump tool (apache#4315)
  HDDS-8129. ContainerStateMachine allows two different tasks with the same container id running in parallel. (apache#4370)
  HDDS-8119. Remove loosely related AutoCloseable from SendContainerOutputStream (apache#4368)
  close db connection (apache#4366)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants