-
Notifications
You must be signed in to change notification settings - Fork 150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: accept empty arrays in struct field lookup #997
Conversation
Fixes apache#992. Empty `pyarrow` arrays are considered falsy, which caused a `ResolveError` for required fields during scan operations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great catch @grobgl
@grobgl Thank you for the fix. Could you add a integration test to guarantee the expected behavior? |
This covers the issue reported in apache#992 where empty scan queries yielded a `ResolveError`. Specifically, this occurred under the following conditions: - a table with an ordered, non-nullable string column - a scan filtering for a non-existing value _within_ the range of the values in that particular column
Thank you, @grobgl. Could you please double-check your test implementation? I removed the fix and the test didn't break. |
Removing the fix does break the test for me. Possibly, this depends on your |
That's strange because I'm running your branch on my machine, so we shouldn't be seeing different behavior. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@grobgl, I apologize for the confusion. I rebuilt the environment from scratch and everything is working correctly now. Thank you for your contribution!
* Fix: accept empty arrays in struct field lookup Fixes #992. Empty `pyarrow` arrays are considered falsy, which caused a `ResolveError` for required fields during scan operations. * Integration test: empty scan on non-nullable ordered string column This covers the issue reported in #992 where empty scan queries yielded a `ResolveError`. Specifically, this occurred under the following conditions: - a table with an ordered, non-nullable string column - a scan filtering for a non-existing value _within_ the range of the values in that particular column * Lint (add missing newline)
* Fix: accept empty arrays in struct field lookup Fixes #992. Empty `pyarrow` arrays are considered falsy, which caused a `ResolveError` for required fields during scan operations. * Integration test: empty scan on non-nullable ordered string column This covers the issue reported in #992 where empty scan queries yielded a `ResolveError`. Specifically, this occurred under the following conditions: - a table with an ordered, non-nullable string column - a scan filtering for a non-existing value _within_ the range of the values in that particular column * Lint (add missing newline)
Fixes #992.
Empty
pyarrow
arrays are considered falsy, which caused aResolveError
for required fields during scan operations.