HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily #1369

maobaolong · 2020-09-01T00:37:52Z

What changes were proposed in this pull request?

Provide a way to get the default value and key of java-based-configuration easily.

What is the link to the Apache JIRA

HDDS-4104

How was this patch tested?

This PR with the related unit tests.

…ed-configuration easily

maobaolong · 2020-09-01T08:44:12Z

@elek Thanks to create the Java-based-Configuration framework, it is better than the hadoop configuration, but sometime, I want to get the default value or key name for a config variable, so I submit this PR, hope this can supply a way to use Java-based-Configuration framework conveniently.

adoroszlai

Thanks @maobaolong for working on this. I would like to propose some minor changes and further tests.

...p-hdds/config/src/test/java/org/apache/hadoop/hdds/conf/TestConfigurationReflectionUtil.java

hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationReflectionUtil.java

Co-authored-by: Doroszlai, Attila <[email protected]>

maobaolong · 2020-09-02T16:23:20Z

@adoroszlai Thanks for your review and the good suggestion, I push new commits and addressed the comments inline.
PTAL.

adoroszlai

Thanks @maobaolong for updating the patch, LGTM. Can you check if change in hadoop-ozone/dev-support/intellij/ozone-site.xml is intentional?

hadoop-ozone/dev-support/intellij/ozone-site.xml

codecov-commenter · 2020-09-02T17:49:44Z

Codecov Report

Merging #1369 into master will decrease coverage by 0.19%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master    #1369      +/-   ##
============================================
- Coverage     75.32%   75.13%   -0.20%     
+ Complexity    10502    10501       -1     
============================================
  Files           990      990              
  Lines         50757    50867     +110     
  Branches       4951     4959       +8     
============================================
- Hits          38233    38219      -14     
- Misses        10158    10258     +100     
- Partials       2366     2390      +24

Impacted Files	Coverage Δ	Complexity Δ
.../hadoop/hdds/conf/ConfigurationReflectionUtil.java	`80.62% <100.00%> (+4.20%)`	`40.00 <11.00> (+11.00)`
...hdds/scm/container/CloseContainerEventHandler.java	`72.41% <0.00%> (-17.25%)`	`6.00% <0.00%> (ø%)`
.../hadoop/ozone/recon/scm/ReconContainerManager.java	`83.05% <0.00%> (-13.56%)`	`13.00% <0.00%> (-1.00%)`
...ache/hadoop/ozone/om/codec/S3SecretValueCodec.java	`90.90% <0.00%> (-9.10%)`	`3.00% <0.00%> (-1.00%)`
...on/scm/ReconIncrementalContainerReportHandler.java	`60.00% <0.00%> (-7.50%)`	`4.00% <0.00%> (ø%)`
...va/org/apache/hadoop/hdds/utils/db/RDBMetrics.java	`92.85% <0.00%> (-7.15%)`	`14.00% <0.00%> (-1.00%)`
...doop/hdds/scm/container/ContainerStateManager.java	`81.67% <0.00%> (-6.88%)`	`32.00% <0.00%> (-3.00%)`
...rg/apache/hadoop/hdds/scm/server/SCMCertStore.java	`38.23% <0.00%> (-5.89%)`	`5.00% <0.00%> (-1.00%)`
.../org/apache/hadoop/hdds/scm/XceiverClientGrpc.java	`68.12% <0.00%> (-5.82%)`	`38.00% <0.00%> (ø%)`
...p/ozone/security/OzoneDelegationTokenSelector.java	`72.22% <0.00%> (-5.56%)`	`7.00% <0.00%> (-1.00%)`
... and 33 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d34ab29...67c37eb. Read the comment docs.

maobaolong · 2020-09-02T23:32:47Z

@adoroszlai PTAL.

adoroszlai

Thanks @maobaolong for removing unnecessary changes.

hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationReflectionUtil.java

…onfigurationReflectionUtil.java Co-authored-by: Doroszlai, Attila <[email protected]>

maobaolong · 2020-09-03T16:00:23Z

@adoroszlai Thanks for your help during the reviewing.

maobaolong · 2020-09-04T02:57:09Z

@adoroszlai Thanks to trigger the CI check for me.

maobaolong · 2020-09-15T00:57:44Z

@elek Please take a look at this PR, thanks.

adoroszlai · 2020-09-17T09:25:38Z

Thanks @maobaolong for the contribution. Merged to master.

maobaolong · 2020-09-17T11:10:51Z

@adoroszlai Thanks you merge this PR.

* master: (47 commits) HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily (apache#1369) HDDS-4250. Fix wrong logger name (apache#1429) HDDS-4244. Container deleted wrong replica cause mis-replicated. (apache#1423) HDDS-4053. Volume space: add quotaUsageInBytes and update it when write and delete key. (apache#1296) HDDS-4210. ResolveBucket during checkAcls fails. (apache#1398) HDDS-4075. Retry request on different OM on AccessControlException (apache#1303) HDDS-4166. Documentation index page redirects to the wrong address (apache#1372) HDDS-4039. Reduce the number of fields in hdds.proto to improve performance (apache#1289) HDDS-4155. Directory and filename can end up with same name in a path. (apache#1361) HDDS-3927. Rename Ozone OM,DN,SCM runtime options to conform to naming conventions (apache#1401) HDDS-4119. Improve performance of the BufferPool management of Ozone client (apache#1336) HDDS-4217.Remove test TestOzoneContainerRatis (apache#1408) HDDS-4218.Remove test TestRatisManager (apache#1409) HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE. (apache#1337) HDDS-4228: add field 'num' to ALLOCATE_BLOCK of scm audit log. (apache#1413) HDDS-4196. Add an endpoint in Recon to query Prometheus (apache#1390) HDDS-4211. [OFS] Better owner and group display for listing Ozone volumes and buckets (apache#1397) HDDS-4150. recon.api.TestEndpoints test is flaky (apache#1396) HDDS-4170 - Fix typo in method description. (apache#1406) HDDS-4064. Show container verbose info with verbose option (apache#1290) ...

…ponse * HDDS-4122-quota-attempt2: (51 commits) Remove redundant check status calls in children of AbstractOMOpenKeyDeleteRequest Remove unused inports and fix super constructor calls Move common volume byte usage update code to AbstractOMKeyDeleteResponse Add volume quota update to open key delete response, and group duplicate code HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily (apache#1369) HDDS-4250. Fix wrong logger name (apache#1429) HDDS-4244. Container deleted wrong replica cause mis-replicated. (apache#1423) HDDS-4053. Volume space: add quotaUsageInBytes and update it when write and delete key. (apache#1296) HDDS-4210. ResolveBucket during checkAcls fails. (apache#1398) HDDS-4075. Retry request on different OM on AccessControlException (apache#1303) HDDS-4166. Documentation index page redirects to the wrong address (apache#1372) HDDS-4039. Reduce the number of fields in hdds.proto to improve performance (apache#1289) HDDS-4155. Directory and filename can end up with same name in a path. (apache#1361) HDDS-3927. Rename Ozone OM,DN,SCM runtime options to conform to naming conventions (apache#1401) HDDS-4119. Improve performance of the BufferPool management of Ozone client (apache#1336) HDDS-4217.Remove test TestOzoneContainerRatis (apache#1408) HDDS-4218.Remove test TestRatisManager (apache#1409) HDDS-4129. change MAX_QUOTA_IN_BYTES to Long.MAX_VALUE. (apache#1337) HDDS-4228: add field 'num' to ALLOCATE_BLOCK of scm audit log. (apache#1413) HDDS-4196. Add an endpoint in Recon to query Prometheus (apache#1390) ...

HDDS-4104. Provide a way to get the default value and key of java-bas…

1fc00a3

…ed-configuration easily

adoroszlai requested changes Sep 1, 2020

View reviewed changes

Apply suggestions from code review

747c925

Co-authored-by: Doroszlai, Attila <[email protected]>

maobaolong force-pushed the HDDS-4104 branch 3 times, most recently from ac87b28 to 957cb5f Compare September 2, 2020 16:21

maobaolong requested a review from adoroszlai September 2, 2020 16:23

maobaolong added 2 commits September 3, 2020 00:23

add tests.

957cb5f

Fix style

67c37eb

adoroszlai reviewed Sep 2, 2020

View reviewed changes

hadoop-ozone/dev-support/intellij/ozone-site.xml Outdated Show resolved Hide resolved

maobaolong requested a review from adoroszlai September 2, 2020 23:32

revert ozone-site.xml and rename testcase.

e409d54

maobaolong closed this Sep 3, 2020

maobaolong reopened this Sep 3, 2020

maobaolong closed this Sep 3, 2020

maobaolong reopened this Sep 3, 2020

adoroszlai approved these changes Sep 3, 2020

View reviewed changes

adoroszlai reviewed Sep 3, 2020

View reviewed changes

hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/ConfigurationReflectionUtil.java Outdated Show resolved Hide resolved

Update hadoop-hdds/config/src/main/java/org/apache/hadoop/hdds/conf/C…

a497618

…onfigurationReflectionUtil.java Co-authored-by: Doroszlai, Attila <[email protected]>

trigger new CI check

dc210f5

adoroszlai requested a review from elek September 4, 2020 06:50

adoroszlai merged commit 079ee7f into apache:master Sep 17, 2020

HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily #1369

HDDS-4104. Provide a way to get the default value and key of java-based-configuration easily #1369

Uh oh!

Conversation

maobaolong commented Sep 1, 2020

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

maobaolong commented Sep 1, 2020

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maobaolong commented Sep 2, 2020

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov-commenter commented Sep 2, 2020

Codecov Report

Uh oh!

maobaolong commented Sep 2, 2020

Uh oh!

adoroszlai left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

maobaolong commented Sep 3, 2020

Uh oh!

maobaolong commented Sep 4, 2020

Uh oh!

maobaolong commented Sep 15, 2020

Uh oh!

adoroszlai commented Sep 17, 2020

Uh oh!

maobaolong commented Sep 17, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants