Skip to content

misc: Move IndexJoinNode to SPI#26218

Closed
feilong-liu wants to merge 1 commit intoprestodb:masterfrom
feilong-liu:index_join_connector
Closed

misc: Move IndexJoinNode to SPI#26218
feilong-liu wants to merge 1 commit intoprestodb:masterfrom
feilong-liu:index_join_connector

Conversation

@feilong-liu
Copy link
Contributor

Description

As title, move IndexJoinNode to SPI so that it's available in ApplyConnectorOptimizer

Motivation and Context

As title

Impact

In description.

Test Plan

Easy change, existing unit tests

Contributor checklist

  • Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== NO RELEASE NOTE ==

@prestodb-ci prestodb-ci added the from:Meta PR from Meta label Oct 2, 2025
@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Oct 2, 2025

Reviewer's Guide

Refactored IndexJoinNode into the SPI module and updated all dependent code, serialization, and optimizers to recognize the relocated class, while replacing Guava ImmutableList usage with java.util unmodifiableList.

Class diagram for relocated IndexJoinNode in SPI

classDiagram
    class PlanNode {
        <<abstract>>
        +List<VariableReferenceExpression> getOutputVariables()
        +List<PlanNode> getSources()
        +<R, C> R accept(PlanVisitor<R, C> visitor, C context)
    }
    class JoinType
    class RowExpression
    class VariableReferenceExpression
    class IndexJoinNode {
        +JoinType type
        +PlanNode probeSource
        +PlanNode indexSource
        +List<Pair<VariableReferenceExpression, VariableReferenceExpression>> criteria
        +Optional<RowExpression> filter
        +Optional<VariableReferenceExpression> probeHashVariable
        +Optional<VariableReferenceExpression> indexHashVariable
        +List<VariableReferenceExpression> getLookupVariables()
        +List<PlanNode> getSources()
        +List<VariableReferenceExpression> getOutputVariables()
        +<R, C> R accept(PlanVisitor<R, C> visitor, C context)
    }
    PlanNode <|-- IndexJoinNode
    IndexJoinNode o-- JoinType
    IndexJoinNode o-- PlanNode : probeSource
    IndexJoinNode o-- PlanNode : indexSource
    IndexJoinNode o-- RowExpression : filter
    IndexJoinNode o-- VariableReferenceExpression : probeHashVariable
    IndexJoinNode o-- VariableReferenceExpression : indexHashVariable
    IndexJoinNode o-- VariableReferenceExpression : criteria
Loading

Class diagram for PlanVisitor with new visitIndexJoin method

classDiagram
    class PlanVisitor {
        +R visitJoin(JoinNode node, C context)
        +R visitIndexJoin(IndexJoinNode node, C context)
        +R visitSemiJoin(SemiJoinNode node, C context)
        +R visitPlan(PlanNode node, C context)
    }
    class IndexJoinNode
    PlanVisitor ..> IndexJoinNode : visitIndexJoin
Loading

File-Level Changes

Change Details Files
Relocate IndexJoinNode from planner package to SPI
  • Changed package declaration and imports in IndexJoinNode
  • Updated class to extend PlanNode instead of InternalPlanNode
  • Moved Java source from presto-main-base to presto-spi
  • Adjusted YML JavaClasses entries to reference SPI path
presto-spi/src/main/java/com/facebook/presto/spi/plan/IndexJoinNode.java
presto-main-base/src/main/java/com/facebook/presto/sql/planner/plan/IndexJoinNode.java
presto-native-execution/presto_cpp/presto_protocol/core/presto_protocol_core.yml
presto-native-execution/presto_cpp/presto_protocol/presto_protocol.yml
Replace Guava ImmutableList with unmodifiableList
  • Replaced ImmutableList.copyOf with Collections.unmodifiableList
  • Rewrote getSources and getOutputVariables to build via ArrayList
  • Swapped com.google.common import to java.util
presto-spi/src/main/java/com/facebook/presto/spi/plan/IndexJoinNode.java
Update C++ protocol serialization for IndexJoinNode
  • Changed @type and type keys to ".IndexJoinNode"
  • Updated to_json/from_json in core.cpp and header
  • Adjusted YAML entries for serialization keys
presto-native-execution/presto_cpp/presto_protocol/core/presto_protocol_core.cpp
presto-native-execution/presto_cpp/presto_protocol/core/presto_protocol_core.h
presto-native-execution/presto_cpp/presto_protocol/core/presto_protocol_core.yml
presto-native-execution/presto_cpp/presto_protocol/presto_protocol.yml
Support IndexJoinNode in visitor and optimizer
  • Added visitIndexJoin to PlanVisitor
  • Included IndexJoinNode in ApplyConnectorOptimization node list
  • Updated test import for TestNativeIndexJoinLogicalPlanner
presto-spi/src/main/java/com/facebook/presto/spi/plan/PlanVisitor.java
presto-main-base/src/main/java/com/facebook/presto/sql/planner/optimizations/ApplyConnectorOptimization.java
presto-tests/src/test/java/com/facebook/presto/tests/TestNativeIndexJoinLogicalPlanner.java

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@feilong-liu feilong-liu changed the title Move IndexJoinNode to SPI misc: Move IndexJoinNode to SPI Oct 2, 2025
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@feilong-liu feilong-liu force-pushed the index_join_connector branch from 77ec333 to 985aec0 Compare October 3, 2025 16:52
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove index join from internalPlanVisitor

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should have some sanity check for this. It's very easy to miss.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed from internalPlanVisitor

rschlussel
rschlussel previously approved these changes Oct 3, 2025
hantangwangd
hantangwangd previously approved these changes Oct 3, 2025
Copy link
Member

@hantangwangd hantangwangd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @feilong-liu, lgtm!

@feilong-liu
Copy link
Contributor Author

Thanks @hantangwangd and @rschlussel for reviewing this PR. The prestocpp-linux-build / prestocpp-linux-build-engine (pull_request) CI job has been consistently failing due to disk full error, so I created another PR #26270 following a previous PR #24331 to save some disk space, can I get a review for that PR too? Thanks!

feilong-liu added a commit that referenced this pull request Oct 10, 2025
…inux-build-engine job (#26270)

## Description
Reduce disk consumption for prestocpp-linux-build-engine.
I have a PR #26218 which
consistently fail due to disk full. Similar to
#24331, this PR further remove
files which are not used in the CI build.

## Motivation and Context
As in description.

## Impact
Fix failed CI test

## Test Plan
The CI test completes successfully
https://github.com/prestodb/presto/actions/runs/18389979579/job/52397841783?pr=26270

## Contributor checklist

- [ ] Please make sure your submission complies with our [contributing
guide](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md),
in particular [code
style](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#code-style)
and [commit
standards](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#commit-standards).
- [ ] PR description addresses the issue accurately and concisely. If
the change is non-trivial, a GitHub Issue is referenced.
- [ ] Documented new properties (with its default value), SQL syntax,
functions, or other functionality.
- [ ] If release notes are required, they follow the [release notes
guidelines](https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines).
- [ ] Adequate tests were added if applicable.
- [ ] CI passed.

## Release Notes
Please follow [release notes
guidelines](https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines)
and fill in the release notes below.


If release note is NOT required, use:

```
== NO RELEASE NOTE ==
```
@feilong-liu
Copy link
Contributor Author

Replaced by #26289

imsayari404 pushed a commit to imsayari404/presto that referenced this pull request Oct 13, 2025
…inux-build-engine job (prestodb#26270)

## Description
Reduce disk consumption for prestocpp-linux-build-engine.
I have a PR prestodb#26218 which
consistently fail due to disk full. Similar to
prestodb#24331, this PR further remove
files which are not used in the CI build.

## Motivation and Context
As in description.

## Impact
Fix failed CI test

## Test Plan
The CI test completes successfully
https://github.com/prestodb/presto/actions/runs/18389979579/job/52397841783?pr=26270

## Contributor checklist

- [ ] Please make sure your submission complies with our [contributing
guide](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md),
in particular [code
style](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#code-style)
and [commit
standards](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#commit-standards).
- [ ] PR description addresses the issue accurately and concisely. If
the change is non-trivial, a GitHub Issue is referenced.
- [ ] Documented new properties (with its default value), SQL syntax,
functions, or other functionality.
- [ ] If release notes are required, they follow the [release notes
guidelines](https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines).
- [ ] Adequate tests were added if applicable.
- [ ] CI passed.

## Release Notes
Please follow [release notes
guidelines](https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines)
and fill in the release notes below.


If release note is NOT required, use:

```
== NO RELEASE NOTE ==
```
rschlussel pushed a commit that referenced this pull request Oct 13, 2025
## Description
As title, move IndexJoinNode to SPI so that it's available in
ApplyConnectorOptimizer

The original PR #26218 always run
out of disk space for job `prestocpp-linux-build /
prestocpp-linux-build-engine`, even after multiple attempts to save disk
space #26276 and
#26270 A fresh build, i.e.
without cache for branch, seems to solve this problem, so create a new
PR with the same change.

## Motivation and Context
In description.

## Impact
In description.

## Test Plan
In description.

## Contributor checklist

- [ ] Please make sure your submission complies with our [contributing
guide](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md),
in particular [code
style](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#code-style)
and [commit
standards](https://github.com/prestodb/presto/blob/master/CONTRIBUTING.md#commit-standards).
- [ ] PR description addresses the issue accurately and concisely. If
the change is non-trivial, a GitHub Issue is referenced.
- [ ] Documented new properties (with its default value), SQL syntax,
functions, or other functionality.
- [ ] If release notes are required, they follow the [release notes
guidelines](https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines).
- [ ] Adequate tests were added if applicable.
- [ ] CI passed.

## Release Notes
Please follow [release notes
guidelines](https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines)
and fill in the release notes below.

If release note is NOT required, use:

```
== NO RELEASE NOTE ==
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

from:Meta PR from Meta

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants