Skip to content

Prefer spilling in window fullGroupBuffered when there was at least one spill#27873

Merged
sopel39 merged 2 commits intotrinodb:masterfrom
sopel39:oss/fix_spill_window_and_aggr
Jan 12, 2026
Merged

Prefer spilling in window fullGroupBuffered when there was at least one spill#27873
sopel39 merged 2 commits intotrinodb:masterfrom
sopel39:oss/fix_spill_window_and_aggr

Conversation

@sopel39
Copy link
Copy Markdown
Member

@sopel39 sopel39 commented Jan 7, 2026

When spill happened, prefer spilling rather than merging with memory.
This prevents query node-level OOM when query is spill heavy

Additionally, a minor bug was fixed in spoilable hash aggregation, when removable memory
was not covered to user memory when merge with memory happens

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

## General
* Reduce out-of-memory errors in window queries when spill is enabled. ({issue}`issuenumber`)

Fixes #10596

@sopel39 sopel39 requested a review from raunaqmorarka January 7, 2026 15:19
@cla-bot
Copy link
Copy Markdown

cla-bot bot commented Jan 7, 2026

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Karol Sobczak.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email email@example.com
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

Revocable memory needs to be converted to user memory as producing output stage is no longer revocable.
When spill happened, prefer spilling rather than merging with memory.
This prevents query node-level OOM when query is spill heavy.
@sopel39 sopel39 force-pushed the oss/fix_spill_window_and_aggr branch from d025203 to 8bd42d4 Compare January 7, 2026 15:22
@cla-bot cla-bot bot added the cla-signed label Jan 7, 2026
@sopel39 sopel39 merged commit cc226aa into trinodb:master Jan 12, 2026
98 of 100 checks passed
@sopel39 sopel39 deleted the oss/fix_spill_window_and_aggr branch January 12, 2026 13:47
@github-actions github-actions bot added this to the 480 milestone Jan 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

HashAggregationOperator misreports memory usage after spilling

2 participants