Skip to content

Add support for OFFSET clause#16035

Merged
rongrong merged 4 commits intoprestodb:masterfrom
fgwang7w:offset_clause
Jun 22, 2021
Merged

Add support for OFFSET clause#16035
rongrong merged 4 commits intoprestodb:masterfrom
fgwang7w:offset_clause

Conversation

@fgwang7w
Copy link
Member

@fgwang7w fgwang7w commented May 2, 2021

This PR adds support for the result offset clause in SQL query expressions.

Syntax

<query expression> ::=
  [ <with clause> ] <query expression body>
      [ <order by clause> ] [ <result offset clause> ]

<result offset clause> ::=
  OFFSET <offset row count> { ROW | ROWS }

<offset row count> ::=
  <simple value specification>

The OFFSET node is implemented by transforming it to RowNumber and Filter or FilterProject nodes. An example query plan is as follows:
image

Resolves #14837

Cherry-pick of trinodb/trino#723, trinodb/trino#732, trinodb/trino#746, trinodb/trino#820

Co-authored-by: kasiafi 30203062+kasiafi@users.noreply.github.com

Test plan - Add UT coverage, and test the feature locally

== RELEASE NOTES ==

General Changes
* Add support for the result offset clause in SQL query expressions. This feature can be enabled by setting the session property ``offset_clause_enabled`` or configuration property ``offset-clause-enabled`` to true.

@fgwang7w fgwang7w force-pushed the offset_clause branch 4 times, most recently from 5df5fd4 to a1e60fa Compare May 4, 2021 16:59
@fgwang7w fgwang7w closed this May 4, 2021
@fgwang7w fgwang7w reopened this May 4, 2021
@fgwang7w fgwang7w force-pushed the offset_clause branch 4 times, most recently from 26f021d to 1201c07 Compare May 5, 2021 21:38
@fgwang7w fgwang7w requested review from kaikalur and tdcmeehan May 5, 2021 21:50
@rongrong rongrong requested a review from yingsu00 May 10, 2021 23:28
@rongrong
Copy link
Contributor

Can we make sure we have a "kill switch" for this feature? A configuration to disable it if things go wrong in production.

@fgwang7w
Copy link
Member Author

Can we make sure we have a "kill switch" for this feature? A configuration to disable it if things go wrong in production.

A session-level control switch enable-offset is added

@fgwang7w fgwang7w force-pushed the offset_clause branch 3 times, most recently from 029bd5c to 1d7537b Compare May 17, 2021 16:57
@fgwang7w
Copy link
Member Author

@rongrong @yingsu00 @tdcmeehan please help review the PR, thanks

@tdcmeehan tdcmeehan requested a review from prithvip May 19, 2021 19:34
@yingsu00
Copy link
Contributor

This PR needs a release note. Please refer to https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines to add one in the PR description section. The release bot will automatically gather it from all merged PRs.

@yingsu00
Copy link
Contributor

Let's use the same commit names as the Trino ones so that readers will be easy to reason about the history

Copy link
Contributor

@yingsu00 yingsu00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fgwang7w Hi George, thanks for back porting the PR. I am running out of time and only finished the first part. I'll flush the comments I have so far and finish the rest tomorrow.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isEnableOffset -> isOffsetClauseEnabled

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fgwang7w I don't see this function name changed yet.

Copy link
Contributor

@yingsu00 yingsu00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the unit test TestImplementOffset.java? It should be added in package com.facebook.presto.sql.planner.iterative.rule

Copy link
Contributor

@yingsu00 yingsu00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fgwang7w Would it be possible to add a few plan examples for OFFSET clause in the PR message?

@fgwang7w
Copy link
Member Author

@fgwang7w Would it be possible to add a few plan examples for OFFSET clause in the PR message?

Thank you for reviewing the PR. Please help review the revision. TestImplementOffset is added back, query plan is updated in the PR message session.

@fgwang7w fgwang7w requested a review from yingsu00 May 24, 2021 15:52
@yingsu00
Copy link
Contributor

Commit message 1

I'd prefer changing the message Add support for OFFSET syntax to its original message Add support for OFFSET syntax in grammar and AST

Cherry-pick of trino/#723 This doesn't resolve to the correct http link but to a different commit. Let's use full URL:
Cherry-pick of https://github.com/trinodb/trino/pull/723

Copy link
Contributor

@yingsu00 yingsu00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I strongly believe trinodb/trino#746 should belong to the first commit(parser changes) not the second(planner changes). Would you please also update the commit messages so that the first commit contains 723 and 746 and the second commit contains the rest? Remember to use the full URL so that they resolve to the correct commits. Then add another commit for the documentation. Thanks!

@fgwang7w
Copy link
Member Author

fgwang7w commented Jun 4, 2021

@yingsu00 Thanks for suggestions and review. Please help review the revisions again.

Copy link
Contributor

@yingsu00 yingsu00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fgwang7w Hi George, it looks in good shape. Just two small catches:

  1. The function name isEnableOffset in SystemSessionProperties was still not updated
  2. The commit message for the second commit still has 746 included. Please remove 746 from the commit message.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fgwang7w I don't see this function name changed yet.

@yingsu00
Copy link
Contributor

yingsu00 commented Jun 5, 2021

@rongrong I think this PR is almost good to go (waiting for two small issues being fixed). Do you want to take a second pass? Thanks!

@yingsu00
Copy link
Contributor

yingsu00 commented Jun 8, 2021

@fgwang7w Just discussed with Rongrong. Will you be able to set the session and config properties default to false, and move the 3rd commit to the first?

@fgwang7w
Copy link
Member Author

fgwang7w commented Jun 8, 2021

@yingsu00 @rongrong thanks for the feedback. I have changed the default to disable. So the commits are in following order:

  1. enable parser yet fails at analyzer
  2. implement offset
  3. disable offset
  4. update doc.

Let me know if you have any more feedback. Thanks

@fgwang7w fgwang7w force-pushed the offset_clause branch 2 times, most recently from 0d0b2f5 to 6ed0ef0 Compare June 8, 2021 06:43
fgwang7w and others added 2 commits June 8, 2021 23:35
Cherry-pick of trinodb/trino#723 and trinodb/trino#746

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
Cherry-pick of trinodb/trino#732, trinodb/trino#820

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
fgwang7w and others added 2 commits June 9, 2021 00:31
Cherry-pick of trinodb/trino#750 and trinodb/trino#856

Co-authored-by: kasiafi <30203062+kasiafi@users.noreply.github.com>
@NikhilCollooru
Copy link
Contributor

@rongrong looks like all the comments are addressed. Can you take a look and merge it .

@rongrong rongrong merged commit 13a241b into prestodb:master Jun 22, 2021
@rongrong
Copy link
Contributor

Thanks for your contribution!

@ajaygeorge ajaygeorge mentioned this pull request Jul 7, 2021
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Port OFFSET from prestosql

4 participants