-
Notifications
You must be signed in to change notification settings - Fork 9.2k
HADOOP-17483. Magic committer is enabled by default. #2656
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HADOOP-17483. Magic committer is enabled by default. #2656
Conversation
* core-default.xml updated * CommitConstants updated * All tests which previously enabled the magic commiter now rely on default settings. This helps make sure it is enabled. * Docs cover the switch, mention its enabled and explain why you may want to disable it. Change-Id: I40a24a34d519f412d5669ec7ca1de813ad071625
|
Tested: S3 london Unbuffer test failure; also triggered a related failure in ITestS3AContractStreamIOStatistics Filed: https://issues.apache.org/jira/browse/HADOOP-17499 These are network buffer related; read() calls returning less than the full buffer. |
liuml07
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
the change is indeed simpler than #2637
* core-default.xml updated so that fs.s3a.committer.magic.enabled = true * CommitConstants updated to match * All tests which previously enabled the magic committer now rely on default settings. This helps make sure it is enabled. * Docs cover the switch, mention its enabled and explain why you may want to disable it. Note: this doesn't switch to using the committer -it just enables the path rewriting magic which it depends on. Contributed by Steve Loughran.
|
thanks. merged to 3.3+ |
|
🎊 +1 overall
This message was automatically generated. |
…che#2656) * core-default.xml updated so that fs.s3a.committer.magic.enabled = true * CommitConstants updated to match * All tests which previously enabled the magic committer now rely on default settings. This helps make sure it is enabled. * Docs cover the switch, mention its enabled and explain why you may want to disable it. Note: this doesn't switch to using the committer -it just enables the path rewriting magic which it depends on. Contributed by Steve Loughran. this patch updates core-default.xml; the 7.1-maint patch did not as the magic committer was already enabled there. Change-Id: I409a6c444fae843046a700a5f9e645033bdb411a
default settings. This helps make sure it is enabled.
want to disable it.
Change-Id: I40a24a34d519f412d5669ec7ca1de813ad071625