-
Notifications
You must be signed in to change notification settings - Fork 3k
fix: coerce UUID to String in readable_metrics to avoid ClassCastException in Spark #13087
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: coerce UUID to String in readable_metrics to avoid ClassCastException in Spark #13087
Conversation
singhpk234
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thank you for the fix @kadai0308 ! can you please add an UT for this as well
@Fokko |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
|
@Fokko |
|
I'm having the same problem when I run |
|
@Fokko @singhpk234 Just a friendly reminder about this PR. |
Fokko
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes sense to me @kadai0308 👍
|
This PR broke one of the tests ( I understand this change is mainly intended for Spark, but it will impact Java API users as well. I had to introduce a redundant cast into our connector. Ideally, Spark specific change should be done in Spark module, not API. |
|
@ebyhr Good catch, and I agree that we should fix this on the Spark side 👍 |
This reverts commit 7fefc46.
Spark expects all StringType fields to be castable to CharSequence, but Iceberg's
readable_metrics lower_bound/upper_bound may decode to java.util.UUID for UUID-typed
columns. This causes a runtime ClassCastException when Spark tries to read those
metrics as UTF8String.
This commit fixes the issue by converting UUID values to string when generating
readable metric values for Spark metadata tables.
Closes: #13077 (comment)