-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Push down filters below Unnest
in sub queries
#10935
Comments
For this kind of query Unnest with recursive, and max_depth
|
I think this is related #10660 |
Thanks for the pointers @duongcongtoai |
@ahirner
Given that, it might not be necessary to support pushing down outer filter |
Yes. The problem is that one can't move the filter to the inner query (views) or one shouldn't have to with other common sql constructs (sub queries, ctes). |
I think we can rewrite Unnest plan in push down filter |
PR to do so: #10974 |
I confirm that 1b396c4 fixes our more involved case in practice. |
Thanks for the swift action. I just found out this second, that the change introduces a new issue when filtering on an unnested struct column ( I'll file a new issue. The reproducer is less trivial as here. |
Added issue that's quite minimal but maybe not entirely minimal: #10990 |
Thank you @jayzhan211 for the quick response |
Is your feature request related to a problem or challenge?
Currently where clauses on a sub query are not pushed below
Unnest
expressions. In conjunction with some other limitations, this makes composing queries withunnest()
not ideal.In
39.0.0
cli:We get equal results with or w/o a subquery:
Filter
andFilterExec
are above projections withunnest
, instead of below.Ok:
Problematic:
Describe the solution you'd like
FilterExec
is pushed down to tables that support them.Describe alternatives you've considered
A workaround is trying to not use subqueries. However, then one cannot unest structs with multiple lists, or unnest a column more than once. Those are also unsupported.
(note: a real case would be something like
select unnest(named_struct)['foo'] as foo, unnest(named_struct)['bar'] as bar
).Additional context
Maybe related: #5364
The text was updated successfully, but these errors were encountered: