Skip to content

Conversation

@emilysun201309
Copy link
Contributor

@emilysun201309 emilysun201309 commented Jun 20, 2025

Summary:
Pre-determine the output buffer size for sort keys by applying BinarySortableSerializer::serializedSizeInBytes on the input data.
Use both the valueOutputBufferSize and keyOutputBufferSize to determine batch size
Allocate the buffer size needed and avoid resizing the buffer

Reviewed By: xiaoxmeng

Differential Revision: D76008316

== RELEASE NOTES ==
Prestissimo (Native Execution) Changes
* Improve memory usage in PartitionAndSerialize Operator by pre-determine the serialized byte size of a given sort key at 'rowId'. This allows PartitionAndSerialize Operator to pre-allocated the exact output buffer size needed for serialization, avoid wasted memory allocation. User should expect lower memory usage and up to 20% runtime increase when serializing a sort key. 

…ermined size

Summary:
Pre-determine the output buffer size for sort keys by doing a first pass on the input data. 
Use both the  valueOutputBufferSize and keyOutputBufferSize to determine batch size
Allocate the buffer size needed and avoid resizing the buffer

Reviewed By: xiaoxmeng

Differential Revision: D76008316
@emilysun201309 emilysun201309 requested a review from a team as a code owner June 20, 2025 16:51
@prestodb-ci prestodb-ci added the from:Meta PR from Meta label Jun 20, 2025
@facebook-github-bot
Copy link
Collaborator

This pull request was exported from Phabricator. Differential Revision: D76008316

@steveburnett
Copy link
Contributor

Thanks for the release note entry!

  • Please revise the entry to begin with a keyword in the Order of changes in the Release Notes Guidelines such as Fix or Improve.
  • Can you describe this PR's effect from a Presto user's perspective? What would a Presto user notice from the change in this PR? (This may help identify how to address the first point.)

@xiaoxmeng xiaoxmeng merged commit 09f555c into prestodb:master Jun 20, 2025
111 of 116 checks passed
@prestodb-ci prestodb-ci mentioned this pull request Jul 28, 2025
6 tasks
@yhwang
Copy link
Member

yhwang commented Aug 18, 2025

Hi @emilysun201309, I am working on the release note of this PR. Can you summarize the note and use user-understandable wording? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants