feat(optimizer): Native TopNRank optimization by aditi-pandit · Pull Request #24138 · prestodb/presto

aditi-pandit · 2024-11-25T10:54:29Z

Description

https://docs.google.com/document/d/1WQfNigR9bVrbM-PqY7F0mswcetN_tdNahzD9ENye-Q0/edit?tab=t.0

Motivation and Context

TPC-DS Q67

== RELEASE NOTES ==

General Changes
* Add Window filter pushdown in native engine for rank and dense_rank functions. Use session property `optimizer.optimize-top-n-rank` to enable the rewrite.

BryanCutler

LGTM just a couple questions

BryanCutler · 2024-12-10T19:14:48Z

presto-main/src/main/java/com/facebook/presto/sql/planner/plan/TopNRowNumberNode.java

+            @JsonProperty("rankingType") RankingFunction rankingFunction,
            @JsonProperty("rowNumberVariable") VariableReferenceExpression rowNumberVariable,
-            @JsonProperty("maxRowCountPerPartition") int maxRowCountPerPartition,
+            @JsonProperty("maxRowCountPerPartition") int maxRankPerPartition,


Should the json property be changed too or is that not possible without breaking compat?

BryanCutler · 2024-12-11T17:53:47Z

...e-execution/src/test/java/com/facebook/presto/nativeworker/PrestoNativeQueryRunnerUtils.java

                        // Write config file
                        String configProperties = format("discovery.uri=%s%n" +
                                "presto.version=testversion%n" +
+                                "native-execution-enabled=true" +


does this need a %n? I'm guessing that's to give it a newline but not sure

steveburnett · 2025-01-09T15:37:29Z

Thank you for the release note! Some nits of phrasing and formatting suggestions.

== RELEASE NOTES ==

Prestissimo (Native Execution) Changes
* Improve Window filter pushdown in Prestissimo engine for ``rank`` and ``dense_rank`` functions. :pr:`24138`

steveburnett · 2025-03-25T19:45:19Z

One more nit about the release note entry: we've automated the link to the PR so you don't have to add it manually anymore.

== RELEASE NOTES ==

Prestissimo (Native Execution) Changes
* Improve Window filter pushdown in Prestissimo engine for ``rank`` and ``dense_rank`` functions.

Summary: Design doc : https://docs.google.com/document/d/1WQfNigR9bVrbM-PqY7F0mswcetN_tdNahzD9ENye-Q0/edit?usp=sharing https://github.com/facebookincubator/velox/discussions/9404 e2e Presto PR (with changes in the Presto optimizer as well) prestodb/presto#24138 Latency for SF1K TPC-DS Q67 fell from 399s to 146s with this change. Pull Request resolved: #14141 Reviewed By: pratikpugalia Differential Revision: D85763891 Pulled By: kevinwilfong fbshipit-source-id: 122bc6d60a4cfbf90c7921ebb1df28ef6edcc0a0

psantos-denodo · 2026-01-21T09:23:52Z

Hi! it looks this is ready to merge, just missing one failing check, is that right? if that's the case could you re-run it? it looks this feature can make a significant improvement and it would be great to merge it soon :)

kaikalur · 2026-01-21T18:08:54Z

Hmm - looks like we already have: optimize_top_n_row_number that does it in the plan? What is this for?

aditi-pandit · 2026-01-21T22:20:08Z

Hi! it looks this is ready to merge, just missing one failing check, is that right? if that's the case could you re-run it? it looks this feature can make a significant improvement and it would be great to merge it soon :)

Hi @psantos-denodo : I have one failure to debug and will prioritize submitting this change.

aditi-pandit · 2026-01-21T22:22:14Z

Hmm - looks like we already have: optimize_top_n_row_number that does it in the plan? What is this for?

@kaikalur : optimize_top_n_row_number was only for queries with row_number. This PR added support for rank and dense_rank window functions pushdown in that same operator. This had Velox runtime code changes as well. The Velox team wanted to disable the optimization by default (optimize_top_n_row_number is true) and so I added a new config for it.

github-actions · 2026-01-23T20:28:29Z

Codenotify: Notifying subscribers in CODENOTIFY files for diff fc589a5...a43be71.

No notifications.

hantangwangd

Thanks @aditi-pandit, a couple of little nits.

presto-docs/src/main/sphinx/presto_cpp/properties-session.rst

...n-base/src/main/java/com/facebook/presto/sql/planner/optimizations/WindowFilterPushDown.java

aditi-pandit · 2026-02-02T00:02:05Z

Submit only after facebookincubator/velox#16190 is updated in Velox submodule.

aditi-pandit · 2026-02-02T00:03:35Z

Hi! it looks this is ready to merge, just missing one failing check, is that right? if that's the case could you re-run it? it looks this feature can make a significant improvement and it would be great to merge it soon :)

Hi @psantos-denodo : I have one failure to debug and will prioritize submitting this change.

Hi @psantos-denodo : I have debugged the failure and have the fix in facebookincubator/velox#16190. We can submit this PR after that one is merged in Velox and the module updated.

psantos-denodo · 2026-02-02T09:16:29Z

Hi! it looks this is ready to merge, just missing one failing check, is that right? if that's the case could you re-run it? it looks this feature can make a significant improvement and it would be great to merge it soon :)

Hi @psantos-denodo : I have one failure to debug and will prioritize submitting this change.

Hi @psantos-denodo : I have debugged the failure and have the fix in facebookincubator/velox#16190. We can submit this PR after that one is merged in Velox and the module updated.

That's great news, thanks for sharing!

aditi-pandit · 2026-02-11T21:41:25Z

@hantangwangd @aaneja @tdcmeehan : This PR is finally clean.

Would be great to have a round of review.

Really appreciate the help.

steveburnett

LGTM! (docs)

hantangwangd

Thanks @aditi-pandit, lgtm!

aditi-pandit · 2026-02-12T00:51:04Z

Thanks @hantangwangd for all the detailed reviews.

## Description https://docs.google.com/document/d/1WQfNigR9bVrbM-PqY7F0mswcetN_tdNahzD9ENye-Q0/edit?tab=t.0 ## Motivation and Context TPC-DS Q67 ``` == RELEASE NOTES == General Changes * Add Window filter pushdown in native engine for rank and dense_rank functions. Use session property `optimizer.optimize-top-n-rank` to enable the rewrite. ```

aditi-pandit requested review from a team, feilong-liu, jaystarshot and vivek-bharathan as code owners November 25, 2024 10:54

aditi-pandit requested a review from presto-oss November 25, 2024 10:54

prestodb-ci added the from:IBM PR from IBM label Nov 25, 2024

prestodb-ci requested review from a team, BryanCutler and imjalpreet and removed request for a team November 25, 2024 10:54

aditi-pandit force-pushed the topn branch from e9fb577 to 99600e4 Compare November 25, 2024 13:27

aditi-pandit mentioned this pull request Nov 25, 2024

(Do not review) feat: TopNRank optimization facebookincubator/velox#11554

Closed

aditi-pandit force-pushed the topn branch 2 times, most recently from b45c503 to ff8a3b5 Compare November 27, 2024 06:29

BryanCutler reviewed Dec 11, 2024

View reviewed changes

aditi-pandit force-pushed the topn branch from e835e9c to aac1159 Compare January 9, 2025 00:20

aditi-pandit force-pushed the topn branch from 56c6d54 to e9246e6 Compare January 10, 2025 02:59

aaneja self-requested a review January 14, 2025 04:09

aditi-pandit mentioned this pull request Feb 11, 2025

fix(iceberg): Date partition value parse issue facebookincubator/velox#12126

Closed

aditi-pandit force-pushed the topn branch 3 times, most recently from a63a0e9 to 2118f4c Compare March 25, 2025 19:33

aditi-pandit force-pushed the topn branch 2 times, most recently from 35f18b2 to 4b96592 Compare March 25, 2025 20:06

aditi-pandit force-pushed the topn branch from 4b96592 to 5c1610d Compare April 4, 2025 04:41

aditi-pandit mentioned this pull request Sep 24, 2025

feat: Add TopNRank optimization facebookincubator/velox#14141

Closed

aditi-pandit dismissed steveburnett’s stale review via 3502832 January 23, 2026 20:25

aditi-pandit force-pushed the topn branch from e161ed7 to 3502832 Compare January 23, 2026 20:25

tdcmeehan self-assigned this Jan 23, 2026

hantangwangd reviewed Jan 25, 2026

View reviewed changes

presto-docs/src/main/sphinx/presto_cpp/properties-session.rst Outdated Show resolved Hide resolved

...n-base/src/main/java/com/facebook/presto/sql/planner/optimizations/WindowFilterPushDown.java Outdated Show resolved Hide resolved

aditi-pandit force-pushed the topn branch 2 times, most recently from 3660237 to 6303d65 Compare February 1, 2026 23:55

aditi-pandit changed the title ~~feat(optimizer): Native TopNRank optimization~~ feat(optimizer): Native TopNRank optimization (needs Velox fix) Feb 2, 2026

feat(native): Add TopNRowNumberNode rank function plumbing to Velox

bc06f0b

aditi-pandit force-pushed the topn branch from 6303d65 to 62e118f Compare February 11, 2026 06:38

feat(optimizer): Add TopN Rank pushdown

a43be71

aditi-pandit force-pushed the topn branch from 62e118f to a43be71 Compare February 11, 2026 06:44

aditi-pandit changed the title ~~feat(optimizer): Native TopNRank optimization (needs Velox fix)~~ feat(optimizer): Native TopNRank optimization Feb 11, 2026

steveburnett approved these changes Feb 11, 2026

View reviewed changes

hantangwangd approved these changes Feb 12, 2026

View reviewed changes

aditi-pandit merged commit ff462d8 into master Feb 12, 2026
124 of 126 checks passed

aditi-pandit deleted the topn branch February 12, 2026 00:50

This was referenced Mar 31, 2026

docs: Add release notes for 0.297 unix280/presto#51

Closed

docs: Add release notes for 0.297 unix280/presto#52

Open

prestodb-ci mentioned this pull request Apr 1, 2026

docs: Add release notes for 0.297 #27484

Open

15 tasks

Conversation

aditi-pandit commented Nov 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Uh oh!

BryanCutler left a comment

Choose a reason for hiding this comment

Uh oh!

BryanCutler Dec 10, 2024

Choose a reason for hiding this comment

Uh oh!

BryanCutler Dec 11, 2024

Choose a reason for hiding this comment

Uh oh!

steveburnett commented Jan 9, 2025

Uh oh!

steveburnett commented Mar 25, 2025

Uh oh!

psantos-denodo commented Jan 21, 2026

Uh oh!

kaikalur commented Jan 21, 2026

Uh oh!

aditi-pandit commented Jan 21, 2026

Uh oh!

aditi-pandit commented Jan 21, 2026

Uh oh!

github-actions bot commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hantangwangd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

aditi-pandit commented Feb 2, 2026

Uh oh!

aditi-pandit commented Feb 2, 2026

Uh oh!

psantos-denodo commented Feb 2, 2026

Uh oh!

aditi-pandit commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steveburnett left a comment

Choose a reason for hiding this comment

Uh oh!

hantangwangd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aditi-pandit commented Feb 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

aditi-pandit commented Nov 25, 2024 •

edited

Loading

github-actions bot commented Jan 23, 2026 •

edited

Loading

aditi-pandit commented Feb 11, 2026 •

edited

Loading