[HUDI-3272] If mode==ignore && tableExists, do not execute write logic and sync hive
#4632
Conversation
|
@manojpec : Can you review this patch. |
codope
left a comment
codope
left a comment
There was a problem hiding this comment.
@dongkelun Thanks for contributing. Left one comment. Can you also add a test case to cover the scenario of mode == SaveMode.Ignore && tableExists?
| // Handle various save modes | ||
| if (mode == SaveMode.Ignore && tableExists) { | ||
| log.warn(s"hoodie table at $basePath already exists. Ignoring & not performing actual writes.") | ||
| false |
There was a problem hiding this comment.
Wouldn't this method return false and exit from here if mode == SaveMode.Ignore && tableExists?
There was a problem hiding this comment.
It will return false and exit from here, and the same logic is used in HoodieSparkSqlWriter.write.But the original version will not directly return false, and will continue to execute the following code logic
Ok, I will try to add a test case. |
|
@codope Hello, I have added test cases. Can you have a look? |
…gic and sync hive (apache#4632)
4 participants
Tips
What is the purpose of the pull request
If
mode==ignore && tableExists, do not execute write logic and sync hiveBrief change log
mode==ignore && tableExists, do not execute write logic and sync hiveVerify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Committer checklist
Has a corresponding JIRA in PR title & commit
Commit message is descriptive of the change
CI is green
Necessary doc changes done or have another open PR
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.