Skip to content

feat(native): Split IndexLookupJoin stats for IndexSource#27292

Merged
zacw7 merged 1 commit intoprestodb:masterfrom
zacw7:expand-stats
Mar 10, 2026
Merged

feat(native): Split IndexLookupJoin stats for IndexSource#27292
zacw7 merged 1 commit intoprestodb:masterfrom
zacw7:expand-stats

Conversation

@zacw7
Copy link
Copy Markdown
Member

@zacw7 zacw7 commented Mar 8, 2026

IndexLookupJoin embeds IndexSource as a single Velox operator, but Presto's plan has them as two separate logical plan nodes. This change expands IndexLookupJoin's operator stats using its StatsSplitter so that the Presto coordinator sees stats for both plan nodes.

== NO RELEASE NOTE ==

Summary by Sourcery

Enhancements:

  • Add expansion of IndexLookupJoin operator stats using its StatsSplitter so that embedded IndexSource statistics are reported as separate plan node metrics to the Presto coordinator.

@zacw7 zacw7 requested review from a team as code owners March 8, 2026 20:08
@prestodb-ci prestodb-ci added the from:Meta PR from Meta label Mar 8, 2026
@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai bot commented Mar 8, 2026

Reviewer's Guide

Adds an expansion step for operator stats so that IndexLookupJoin, which embeds IndexSource, reports separate stats entries corresponding to Presto’s logical plan nodes.

Class diagram for OperatorStats and expandOperatorStats

classDiagram
  class OperatorStats {
    +string operatorType
    +optional StatsSplitter statsSplitter
  }

  class StatsSplitter {
    +vector~OperatorStats~ operator() (OperatorStats opStats)
  }

  class PrestoTaskHelpers {
    +vector~OperatorStats~ expandOperatorStats(vector~OperatorStats~ operatorStats)
  }

  OperatorStats --> StatsSplitter : uses
  PrestoTaskHelpers ..> OperatorStats : processes
  PrestoTaskHelpers ..> StatsSplitter : invokes when
  note for PrestoTaskHelpers "expandOperatorStats expands IndexLookupJoin stats into multiple OperatorStats entries, including a separate one for IndexSource"
Loading

File-Level Changes

Change Details Files
Introduce an operator-stats expansion helper so IndexLookupJoin emits separate stats for its embedded IndexSource logical node.
  • Add expandOperatorStats helper that iterates over collected OperatorStats and conditionally expands IndexLookupJoin using its statsSplitter functor when present
  • Reserve capacity in the expanded stats vector based on the original operator stats size to avoid reallocations
  • Preserve existing operator stats behavior for all operators other than IndexLookupJoin by passing them through unchanged
presto-native-execution/presto_cpp/main/PrestoTask.cpp

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@tanjialiang
Copy link
Copy Markdown
Contributor

Are there followups for this PR? Maybe some comments on this one:

  • should we add tests to this utility?
  • how is this method used? the PR does not show its callsite.

tanjialiang
tanjialiang previously approved these changes Mar 9, 2026
Copy link
Copy Markdown
Contributor

@tanjialiang tanjialiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks LGTM

xiaoxmeng
xiaoxmeng previously approved these changes Mar 9, 2026
Copy link
Copy Markdown
Contributor

@xiaoxmeng xiaoxmeng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zacw7 LGTM % use split to be consistent in naming as in Velox? Thanks!

@zacw7 zacw7 force-pushed the expand-stats branch 2 times, most recently from 9a64aa5 to 540eb67 Compare March 9, 2026 22:22
@zacw7 zacw7 changed the title feat(native): Expand IndexLookupJoin stats for IndexSource feat(native): Split IndexLookupJoin stats for IndexSource Mar 9, 2026
IndexLookupJoin embeds IndexSource as a single Velox operator, but Presto's plan has them as two separate logical plan nodes. This change expands IndexLookupJoin's operator stats using its StatsSplitter so that the Presto coordinator sees stats for both plan nodes.
@zacw7 zacw7 merged commit b38d89c into prestodb:master Mar 10, 2026
113 of 114 checks passed
@zacw7 zacw7 deleted the expand-stats branch March 10, 2026 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:Meta PR from Meta

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants