-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-4553][SQL] Query for parquet table with string fields in spark sql hive get binary result #3414
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Test build #23738 has started for PR 3414 at commit
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fix this
|
Test build #23740 has started for PR 3414 at commit
|
|
Test build #23738 has finished for PR 3414 at commit
|
|
Test PASSed. |
|
Test build #23740 has finished for PR 3414 at commit
|
|
Test PASSed. |
|
I don't think we want to set this flag to always be true otherwise its not possible to read binary data anymore. A reasonable thing to do would be override the conf when the convertMetastoreParquet code path is taken so at least then we are faithfully mimicking hive's behavior. |
|
Test build #23810 has started for PR 3414 at commit
|
|
Test build #23810 has finished for PR 3414 at commit
|
|
Test PASSed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We shouldn't be mutating configuration when accessing a setting. Also, I think that #3441 is going to solve this problem so perhaps we can close this issue?
|
Ok, if @liancheng fix them, i will close this two PR. |
|
mind closing this manually? or script seems to be missing it. |
|
ok to close this |
Link to #3352