Skip to content

Conversation

@pettyjamesm
Copy link
Member

@pettyjamesm pettyjamesm commented Jul 24, 2025

Description

Releases PagesIndex from Ordering(Grouped)Accumulator and MarkDistinctHash from Distinct(Grouped)Accumulator instances after prepareFinal() or evaluateFinal() is called.

This helps mitigate an issue exposed by #25879 - which was that HashAggreationOperator wouldn't track any changes in memory consumption after output started. However, ordered grouped aggregations significantly increase their memory consumption at the beginning of output as a result of transferring the contents of their PagesIndex into their underlying accumulators.

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## General
* Reduce memory required for distinct and ordered grouped aggregations ({issue}`26276`)
* Fix memory tracking for ordered grouped aggregations ({issue}`26276`)

@cla-bot cla-bot bot added the cla-signed label Jul 24, 2025
@pettyjamesm pettyjamesm force-pushed the release-ordered-accumulator-memory branch from c279cc4 to 7c33525 Compare July 24, 2025 16:11
@pettyjamesm pettyjamesm requested a review from dain July 24, 2025 16:14
@pettyjamesm pettyjamesm changed the title Release PagesIndex memory in OrderingAccumulators Release memory eagerly in ordering and distinct accumulators Jul 24, 2025
@pettyjamesm pettyjamesm force-pushed the release-ordered-accumulator-memory branch from 7c33525 to 748ce30 Compare July 24, 2025 16:43
@pettyjamesm pettyjamesm marked this pull request as ready for review July 24, 2025 17:18
@pettyjamesm pettyjamesm merged commit f574832 into trinodb:master Jul 24, 2025
96 checks passed
@pettyjamesm pettyjamesm deleted the release-ordered-accumulator-memory branch July 24, 2025 17:41
@github-actions github-actions bot added this to the 477 milestone Jul 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants