HADOOP-19763: Remove deprecated properties from core-default.xml #8148
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
@steveloughran , in HADOOP-19597, you expressed concerns about |
1 task
|
💔 -1 overall
This message was automatically generated. |
slfan1989
approved these changes
Dec 29, 2025
zabetak
reviewed
Dec 29, 2025
Member
zabetak
left a comment
zabetak
left a comment
There was a problem hiding this comment.
Would it make sense to add a unit test (maybe in org.apache.hadoop.conf.TestConfiguration or org.apache.hadoop.conf.TestConfigurationDeprecation) that loads core-default.xml and validates that it does not contain deprecated properties? Apart from verifying that the current PR is complete it could prevent someone from adding deprecated props in this file in the future.
Somewhat off-topic but ideally all xxx-default.xml files should be auto-generated via code/build scripts and not be manually curated. If that was the case we could avoid having this kind of changes now and in the future.
cnauroth
added a commit
to cnauroth/hadoop
that referenced
this pull request
Dec 29, 2025
Closes apache#8148 Signed-off-by: Shilun Fan <[email protected]> Reviewed-by: Stamatis Zampetakis <[email protected]>
cnauroth
force-pushed
the
HADOOP-19763
branch
from
484305d to
ca78f32
Compare
Contributor
Author
|
@zabetak , great idea! I pushed up a new test. I'll do the same for YARN in #8149 . I don't have plans right now for adding test coverage of others (hdfs-default.xml, mapred-default.xml, distcp-default.xml, etc.). The same technique could apply there though if anyone wants to add those in the future. |
4 tasks
Closes apache#8148 Signed-off-by: Shilun Fan <[email protected]> Reviewed-by: Stamatis Zampetakis <[email protected]>
cnauroth
force-pushed
the
HADOOP-19763
branch
from
ca78f32 to
50a681e
Compare
|
💔 -1 overall
This message was automatically generated. |
Contributor
Author
|
The build failure is an unrelated pre-existing problem on trunk that I'm trying to fix in #8150 . |
|
💔 -1 overall
This message was automatically generated. |
zabetak
approved these changes
Dec 31, 2025
| @Test | ||
| public void testNoDeprecationsByDefault() throws Exception { | ||
| // Force initialization to make sure deprecations are recorded for later calls to isDeprecated. | ||
| new Configuration(); |
Member
There was a problem hiding this comment.
The test looks good although it can't find all deprecated properties (e.g., io.bytes.per.checksum) cause the respective config classes are not loaded. I logged HADOOP-19769 which may be a more complete solution to avoid deprecated props in config files. Obviously, this PR is already good enough and we don't need to strictly wait for HADOOP-19769. If there is interest about HADOOP-19769 let me know and I will try to finalize the POC.
Contributor
Author
There was a problem hiding this comment.
Thanks, HADOOP-19769 looks like a good idea to me!
cnauroth
commented
Jan 2, 2026
Contributor
Author
cnauroth
left a comment
cnauroth
left a comment
There was a problem hiding this comment.
@slfan1989 , are you still +1 for this? I added a new unit test after your last reviewed.
aajisaka
approved these changes
Jan 5, 2026
Contributor
Author
|
I committed this to trunk. @slfan1989 , @zabetak and @aajisaka , thank you for the reviews. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
Recent trunk changes now produce more assertive warnings about use of deprecated configuration properties. With some of these deprecated properties included in core-default.xml, that means every process launched logs the warnings. Remove them from core-default.xml to avoid these warnings. (Note that this does not take the step of actually removing the code that uses the deprecated properties. They'll continue to work. We just don't want warnings logged by default if the user didn't specifically use these properties.)
How was this patch tested?
Build the distro:
Verify the command above no longer produces deprecation warnings.
For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?