Skip to content

Conversation

@adoroszlai
Copy link
Contributor

What changes were proposed in this pull request?

Missing size suffix in some config properties can prevent Datanode from starting up:

[main] ERROR ozone.HddsDatanodeService: Exception in HddsDatanodeService.
org.apache.hadoop.hdds.conf.ConfigurationException: Failed to inject configuration to OzoneClientConfig.bytesPerChecksum
	at org.apache.hadoop.hdds.conf.ConfigurationReflectionUtil.setField(ConfigurationReflectionUtil.java:146)
	at org.apache.hadoop.hdds.conf.ConfigurationReflectionUtil.injectConfigurationToObject(ConfigurationReflectionUtil.java:103)
	at org.apache.hadoop.hdds.conf.ConfigurationReflectionUtil.injectConfiguration(ConfigurationReflectionUtil.java:73)
	at org.apache.hadoop.hdds.conf.ConfigurationSource.getObject(ConfigurationSource.java:177)
	at org.apache.hadoop.ozone.container.ec.reconstruction.ECReconstructionCoordinator.<init>(ECReconstructionCoordinator.java:128)
	at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.<init>(DatanodeStateMachine.java:223)
	at org.apache.hadoop.ozone.HddsDatanodeService.start(HddsDatanodeService.java:299)
	...
	at org.apache.hadoop.ozone.HddsDatanodeService.main(HddsDatanodeService.java:163)
Caused by: java.lang.IllegalArgumentException: 1024 is not in expected format.Expected format is <number><unit>. e.g. 1000MB
	at org.apache.hadoop.hdds.conf.StorageSize.parse(StorageSize.java:61)
	at org.apache.hadoop.hdds.conf.ConfigType$7.parse(ConfigType.java:121)
	at org.apache.hadoop.hdds.conf.ConfigurationReflectionUtil.setField(ConfigurationReflectionUtil.java:142)
	... 20 more

If suffix is not given, use the unit given for the config property (defaults to bytes).

https://issues.apache.org/jira/browse/HDDS-10423

How was this patch tested?

Added unit test.

CI:
https://github.com/adoroszlai/ozone/actions/runs/8051537545

@adoroszlai adoroszlai self-assigned this Feb 26, 2024
@adoroszlai adoroszlai requested a review from ChenSammi February 26, 2024 20:20
Copy link
Contributor

@smengcl smengcl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. Thanks @adoroszlai !

Copy link
Contributor

@ChenSammi ChenSammi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The patch looks good. Thank you @adoroszlai.

@ChenSammi ChenSammi merged commit 2d77fb4 into apache:master Feb 27, 2024
@adoroszlai adoroszlai deleted the HDDS-10423 branch February 27, 2024 06:59
@adoroszlai
Copy link
Contributor Author

Thanks @ChenSammi, @smengcl for the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants