HDDS-4983. Display key offset for each block in command key info #2051

cchenax · 2021-03-16T11:57:22Z

What changes were proposed in this pull request?

add keyOffset in the key info result

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-4983

How was this patch tested?

example:sh key info /vol1/bucket1/key3

{
  "volumeName" : "vol1",
  "bucketName" : "bucket1",
  "name" : "key3",
  "dataSize" : 644930593,
  "creationTime" : "2021-04-20T09:22:15.206Z",
  "modificationTime" : "2021-04-20T09:22:20.869Z",
  "replicationType" : "RATIS",
  "replicationFactor" : 3,
  "ozoneKeyLocations" : [ {
    "containerID" : 3,
    "localID" : 107544261427200003,
    "length" : 268435456,
    "offset" : 0,
    "keyOffset" : 0
  }, {
    "containerID" : 4,
    "localID" : 107544261427200004,
    "length" : 268435456,
    "offset" : 0,
    "keyOffset" : 268435456
  }, {
    "containerID" : 5,
    "localID" : 107544261427200005,
    "length" : 108059681,
    "offset" : 0,
    "keyOffset" : 536870912
  } ],
  "metadata" : { },
  "fileEncryptionInfo" : null
}

cku328

Thanks @cchenax for working on this.
I have some suggestions for checkstyle not to pass.

cku328 · 2021-03-24T08:10:48Z

hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneKeyLocation.java

+  /**
+   * KeyOffset of this key.
+   */
+  private final long KeyOffset;


Suggested change

private final long KeyOffset;

private final long keyOffset;

use lower camel case: KeyOffset -> keyOffset

cku328 · 2021-03-24T08:14:44Z

hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneKeyLocation.java

   */
  public OzoneKeyLocation(long containerID, long localID,
-                  long length, long offset) {
+                          long length, long offset) {


Nit: seems like an unnecessary adjustment.

cku328 · 2021-03-24T08:17:57Z

hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/OzoneKeyLocation.java

+  /**
+   * Returns the KeyOffset of this Key.
+   */
+  public long getKeyOffset() { return KeyOffset; }


Suggested change

public long getKeyOffset() { return KeyOffset; }

public long getKeyOffset() {

return keyOffset;

}

blocks should have line breaks.

elek · 2021-04-08T12:31:45Z

For the records: jira issue has a conversation about improving the initial implementation:

cchenax · 2021-04-08T12:34:16Z

For the records: jira issue has a conversation about improving the initial implementation:

yes,I will improve it.

elek

Thanks the patch @cchenax

Overall it looks good to me, I checked the class and it's used only on the client side, safe to modify.

I have one question about simplifying the loop.

hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java

…nt/rpc/RpcClient.java Co-authored-by: Elek, Márton <[email protected]>

cchenax · 2021-04-26T09:33:03Z

Thanks the patch @cchenax

Overall it looks good to me, I checked the class and it's used only on the client side, safe to modify.

I have one question about simplifying the loop.

ok,this way is good,thank you very much.

elek

+1, thanks the update @cchenax

…ing-upgrade-master-merge2 * upstream/master: (56 commits) HDDS-2212. Genconf tool should generate config files for secure clust… (apache#1788) HDDS-5166. Remove duplicate assignment of OZONE_OPTS for freon and sh (apache#2195) Revert "HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)" HDDS-4983. Display key offset for each block in command key info (apache#2051) HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177) HDDS-4585. Support bucket acl operation in S3g (apache#1701) HDDS-5153. Decommissioning a dead node should complete immediately (apache#2190) HDDS-5147. Intermittent test failure in TestContainerDeletionChoosingPolicy#testRandomChoosingPolicy (apache#2188) HDDS-5152. Fix Suggested leader in Client. (apache#2189) HDDS-5148. Bump ratis version to 2.1.0-ff8aa66-SNAPSHOT (apache#2184) HDDS-4515. Datanodes should be able to persist and load CRL (apache#2181) HDDS-5060. [SCM HA Security] Make InterSCM grpc channel secure. (apache#2187) HDDS-5051. Ensure failover to suggested leader if any for NotLeaderException. (apache#2141) HDDS-5127. Fix getServiceList when SCM HA is enabled (apache#2173) HDDS-4889. Add simple CI check for docs (apache#2156) HDDS-5131. Use timeout in github actions (apache#2176) HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. (apache#2155) HDDS-5124. Use OzoneConsts.OZONE_TIME_ZONE instead of "GMT" (apache#2166) HDDS-5047. Refactor Pipeline to use ReplicationConfig instead of factor/type (apache#2096) HDDS-5083. Bump version of common-compress (apache#2139) ... Conflicts: hadoop-hdds/common/pom.xml hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConsts.java hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/HddsDatanodeService.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/node/SCMNodeManager.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelineManager.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/SCMStorageConfig.java hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/MiniOzoneClusterImpl.java hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OMStorage.java hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/OzoneManager.java hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/scm/ReconStorageContainerManagerFacade.java

…k-in-auth * HDDS-3698-nonrolling-upgrade: (57 commits) Fix compilation errors afte merge Update javassist in recon pom Fix changes introduced in merge that failed TestSCMNodeManager upgrade tests Fix checkstyle Fix intermittent test failure TestSCMNodeManager#testSetNodeOpStateAndCommandFired after merge Skip scm init default layout version in TestOzoneConfigurationFields HDDS-2212. Genconf tool should generate config files for secure clust… (apache#1788) HDDS-5166. Remove duplicate assignment of OZONE_OPTS for freon and sh (apache#2195) Revert "HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177)" HDDS-4983. Display key offset for each block in command key info (apache#2051) HDDS-5144. Create github check to alert when dependency tree is changed (apache#2177) HDDS-4585. Support bucket acl operation in S3g (apache#1701) HDDS-5153. Decommissioning a dead node should complete immediately (apache#2190) HDDS-5147. Intermittent test failure in TestContainerDeletionChoosingPolicy#testRandomChoosingPolicy (apache#2188) HDDS-5152. Fix Suggested leader in Client. (apache#2189) HDDS-5148. Bump ratis version to 2.1.0-ff8aa66-SNAPSHOT (apache#2184) HDDS-4515. Datanodes should be able to persist and load CRL (apache#2181) HDDS-5060. [SCM HA Security] Make InterSCM grpc channel secure. (apache#2187) HDDS-5051. Ensure failover to suggested leader if any for NotLeaderException. (apache#2141) HDDS-5127. Fix getServiceList when SCM HA is enabled (apache#2173) HDDS-4889. Add simple CI check for docs (apache#2156) HDDS-5131. Use timeout in github actions (apache#2176) HDDS-5103. Fix Install Snapshot Mechanism in SCMStateMachine. (apache#2155) HDDS-5124. Use OzoneConsts.OZONE_TIME_ZONE instead of "GMT" (apache#2166) HDDS-5047. Refactor Pipeline to use ReplicationConfig instead of factor/type (apache#2096) ...

Display key offset for each block in command key info

321e071

adoroszlai changed the title ~~Display key offset for each block in command key info~~ HDDS-4983. Display key offset for each block in command key info Mar 24, 2021

cku328 reviewed Mar 24, 2021

View reviewed changes

Display key offset for each block in command key info

28cf22f

mukul1987 force-pushed the master branch from 520ba00 to 79a9d39 Compare March 25, 2021 05:17

Display key offset for each block in command key info

781743b

cchenax requested a review from cku328 March 25, 2021 09:10

adoroszlai mentioned this pull request Mar 25, 2021

HDDS-4983. Display key offset for each block in command key info #2084

Closed

mukul1987 force-pushed the master branch from 79a9d39 to 520ba00 Compare March 25, 2021 16:05

cchenax force-pushed the HDDS-4983 branch from 7bbd47b to 062ade6 Compare April 15, 2021 11:37

HDDS-4983. Display key offset for each block in command key info

72056ca

cchenax force-pushed the HDDS-4983 branch from 062ade6 to 72056ca Compare April 20, 2021 09:49

cchenaxchen added 3 commits April 20, 2021 17:54

HDDS-4983. Display key offset for each block in command key info

54bb64a

HDDS-4983. Display key offset for each block in command key info

6909fc9

HDDS-4983. Display key offset for each block in command key info

06f279d

elek approved these changes Apr 26, 2021

View reviewed changes

hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java Show resolved Hide resolved

Update hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/clie…

97f22ae

…nt/rpc/RpcClient.java Co-authored-by: Elek, Márton <[email protected]>

retrigger build with empty commit

6611f67

elek approved these changes Apr 29, 2021

View reviewed changes

elek merged commit 10312fb into apache:master Apr 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-4983. Display key offset for each block in command key info #2051

HDDS-4983. Display key offset for each block in command key info #2051

Uh oh!

cchenax commented Mar 16, 2021 •

edited by elek

Loading

Uh oh!

cku328 left a comment

Uh oh!

cku328 Mar 24, 2021

Uh oh!

cku328 Mar 24, 2021

Uh oh!

cku328 Mar 24, 2021

Uh oh!

elek commented Apr 8, 2021

Uh oh!

cchenax commented Apr 8, 2021

Uh oh!

elek left a comment

Uh oh!

Uh oh!

cchenax commented Apr 26, 2021

Uh oh!

elek left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HDDS-4983. Display key offset for each block in command key info #2051

HDDS-4983. Display key offset for each block in command key info #2051

Uh oh!

Conversation

cchenax commented Mar 16, 2021 • edited by elek Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

cku328 left a comment

Choose a reason for hiding this comment

Uh oh!

cku328 Mar 24, 2021

Choose a reason for hiding this comment

Uh oh!

cku328 Mar 24, 2021

Choose a reason for hiding this comment

Uh oh!

cku328 Mar 24, 2021

Choose a reason for hiding this comment

Uh oh!

elek commented Apr 8, 2021

Uh oh!

cchenax commented Apr 8, 2021

Uh oh!

elek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cchenax commented Apr 26, 2021

Uh oh!

elek left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cchenax commented Mar 16, 2021 •

edited by elek

Loading