Skip to content

Conversation

raunaqmorarka
Copy link
Member

@raunaqmorarka raunaqmorarka commented Oct 15, 2025

Description

Engine can provide the sorted Pages directly to the connector.
This avoids code duplication in connector and lets the engine avoid exposing internal details around PagesIndex#sort output

Additional context and related issues

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`issuenumber`)

Summary by Sourcery

Simplify the PageSorter SPI to return an iterator of sorted pages directly and eliminate synthetic address manipulation across the engine and connectors.

Enhancements:

  • Modify PageSorter to return Iterator and remove decodePageIndex and decodePositionIndex methods.
  • Refactor SortBuffer to use the new sorted pages iterator instead of manual PageBuilder merging.
  • Update PagesIndexPageSorter to yield sorted pages via iterator rather than address arrays.

Build:

  • Update revapi configuration in trino-spi pom to remove deprecated PageSorter methods and reflect the signature change.

Tests:

  • Adapt TestPagesIndexPageSorter and BenchmarkPagesIndexPageSorter to consume Iterator and drop synthetic address decoding logic.

Engine can provide the sorted Pages directly to the connector.
This avoids code duplication in connector and lets the engine avoid
exposing internal details around PagesIndex#sort output
@cla-bot cla-bot bot added the cla-signed label Oct 15, 2025
Copy link

sourcery-ai bot commented Oct 15, 2025

Reviewer's Guide

This PR refactors the PageSorter SPI to return sorted Page objects directly instead of synthetic address arrays, removes the decode methods, and updates all engine, connector, test, and benchmark usages to consume the new iterator-based API.

Class diagram for updated PageSorter interface

classDiagram
class PageSorter {
  +Iterator<Page> sort(List<Type> types, List<Page> pages, List<Integer> sortChannels, List<SortOrder> sortOrders, int expectedPositions)
}
class Page
class Type
class SortOrder
PageSorter --> "Iterator<Page>" Page
PageSorter --> Type
PageSorter --> SortOrder
Loading

Class diagram for removed methods from PageSorter

classDiagram
class PageSorter {
  -long[] sort(...)
  -int decodePageIndex(long address)
  -int decodePositionIndex(long address)
}
class Page
PageSorter --> Page
Loading

Class diagram for updated PagesIndexPageSorter implementation

classDiagram
class PagesIndexPageSorter {
  +Iterator<Page> sort(...)
}
class PagesIndex
class Page
PagesIndexPageSorter --> PagesIndex
PagesIndexPageSorter --> "Iterator<Page>" Page
Loading

Class diagram for updated SortBuffer usage

classDiagram
class SortBuffer {
  -PageBuilder pageBuilder
  +void flushTo(Consumer<Page> consumer)
}
class PageSorter
class Page
SortBuffer --> PageSorter
SortBuffer --> Page
Loading

File-Level Changes

Change Details Files
Simplify PageSorter interface
  • Change sort() signature to return Iterator
  • Remove decodePageIndex() and decodePositionIndex() methods
  • Update Revapi config for removed methods and changed return type
core/trino-spi/src/main/java/io/trino/spi/PageSorter.java
core/trino-spi/pom.xml
Implement new interface in PagesIndexPageSorter
  • Return pagesIndex.getSortedPages() from sort()
  • Remove decodePageIndex() and decodePositionIndex() overrides
core/trino-main/src/main/java/io/trino/operator/PagesIndexPageSorter.java
Refactor SortBuffer to consume sorted pages iterator
  • Remove PageBuilder field and manual decode loop
  • Use sortedPages.forEachRemaining(consumer) to emit pages
plugin/trino-hive/src/main/java/io/trino/plugin/hive/util/SortBuffer.java
Update TestPagesIndexPageSorter to use iterator
  • Remove synthetic address assertions and createOutputPages helper
  • Copy pages directly from iterator for comparison
core/trino-main/src/test/java/io/trino/TestPagesIndexPageSorter.java
Adjust benchmark to iterate sorted pages
  • Replace address array usage with Iterator
  • Count positions by iterating pages
core/trino-main/src/test/java/io/trino/BenchmarkPagesIndexPageSorter.java

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@raunaqmorarka raunaqmorarka requested a review from wendigo October 15, 2025 12:28
@github-actions github-actions bot added the hive Hive connector label Oct 15, 2025
Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copy link
Member

@electrum electrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great!

@raunaqmorarka raunaqmorarka merged commit bb5ad10 into master Oct 16, 2025
167 of 168 checks passed
@raunaqmorarka raunaqmorarka deleted the raunaq/sort-clean branch October 16, 2025 04:15
@github-actions github-actions bot added this to the 478 milestone Oct 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed hive Hive connector

Development

Successfully merging this pull request may close these issues.

3 participants