Skip to content

Disallow ORDER BY literal in window functions#15485

Merged
rongrong merged 1 commit intoprestodb:masterfrom
kaikalur:fix_window_orderby_literal
Dec 30, 2020
Merged

Disallow ORDER BY literal in window functions#15485
rongrong merged 1 commit intoprestodb:masterfrom
kaikalur:fix_window_orderby_literal

Conversation

@kaikalur
Copy link
Copy Markdown
Contributor

@kaikalur kaikalur commented Dec 1, 2020

Disallow ORDER BY literal in Window functions as this is currently treated differently for int literals compared to ORDER BY in SELECT where it's treated it as an ordinal.

Test plan - (Please fill in how you tested your changes)
Test included.

== RELEASE NOTES ==

General Changes
* Disallow ORDER BY literals used with Window functions as it's not useful, expensive and most often used wrongly. 

@kaikalur kaikalur requested a review from rongrong December 1, 2020 00:46
@kaikalur kaikalur changed the title Disallow ORDER BY literal (ordinal) in window functions Disallow ORDER BY literal in window functions Dec 1, 2020
Copy link
Copy Markdown
Contributor

@rongrong rongrong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I agree with this change in principle, since this is backward incompatible, we should add a session property that's by default not enabled for the next release, add a warning to say that this feature is not what you think it is, and will be removed in future releases. And then maybe a few release later we would switch the default behavior. Then removing the session property.

@kaikalur
Copy link
Copy Markdown
Contributor Author

kaikalur commented Dec 2, 2020

While I agree with this change in principle, since this is backward incompatible, we should add a session property that's by default not enabled for the next release, add a warning to say that this feature is not what you think it is, and will be removed in future releases. And then maybe a few release later we would switch the default behavior. Then removing the session property.

I agree we should have a way to disable it but I think default should be fail as the fix is trivial or they can just turn on the session property.

@rongrong
Copy link
Copy Markdown
Contributor

rongrong commented Dec 2, 2020

While I agree with this change in principle, since this is backward incompatible, we should add a session property that's by default not enabled for the next release, add a warning to say that this feature is not what you think it is, and will be removed in future releases. And then maybe a few release later we would switch the default behavior. Then removing the session property.

I agree we should have a way to disable it but I think default should be fail as the fix is trivial or they can just turn on the session property.

Overall any unannounced behavior change will never be associated with good user experience. But if you insist the default is fail, at least make it clear in error message how to fix / disable it. And make sure when we deploy it, the default is changed in our internal configuration, unless you already made sure that all affected pipelines are fixed beforehand.

@kaikalur kaikalur force-pushed the fix_window_orderby_literal branch from 083900e to 970f365 Compare December 29, 2020 21:26
@kaikalur
Copy link
Copy Markdown
Contributor Author

kaikalur commented Dec 29, 2020

So added a session param which by default allows this. And so it's a performance warning when enabled, error when not enabled.

@kaikalur kaikalur force-pushed the fix_window_orderby_literal branch from 970f365 to 2e5b7f1 Compare December 29, 2020 22:00
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to call it ordinals or literals? Just pick one! :P

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah - I think literals is more appropriate as that's what we are doing. Fixed.

@kaikalur kaikalur force-pushed the fix_window_orderby_literal branch 2 times, most recently from 9a5c791 to 11d2e53 Compare December 29, 2020 22:39
Copy link
Copy Markdown
Contributor

@rongrong rongrong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more nits. Also I think the release note is also not accurate, please revise. Thanks!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We normally use toImmutableList() rather than Collectors.toList()
We normally format this as

orderBy.getSortItems().stream()
        .filter(...)
        .collect(toImmutableList())

You can use

List<SortItem> constSortItems = window.getOrderBy()
        .ifPresent(orderBy -> orderBy.getSortItems()....)
        .orElse(ImmutableList.of());

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well we already set the context in the first clause. Anyway I fixed it and also removed the unnecessary information.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the description is not accurate.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@kaikalur kaikalur force-pushed the fix_window_orderby_literal branch 5 times, most recently from 1972f2f to 9ed1981 Compare December 30, 2020 15:39
@kaikalur kaikalur force-pushed the fix_window_orderby_literal branch from 9ed1981 to 4f7d7ae Compare December 30, 2020 20:25
@rongrong rongrong merged commit effd0ea into prestodb:master Dec 30, 2020
@caithagoras caithagoras mentioned this pull request Jan 11, 2021
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants