Flush the rate buffer when the slice index changes#138856
Merged
dnhatn merged 1 commit intoelastic:mainfrom Jan 7, 2026
Merged
Flush the rate buffer when the slice index changes#138856dnhatn merged 1 commit intoelastic:mainfrom
dnhatn merged 1 commit intoelastic:mainfrom
Conversation
dnhatn
force-pushed
the
flush-rate-buffer
branch
from
68e242f to
043fc8a
Compare
dnhatn
force-pushed
the
flush-rate-buffer
branch
from
043fc8a to
199bb3c
Compare
dnhatn
added
v9.3.1
:StorageEngine/ES|QL
Collaborator
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
kkrik-es
approved these changes
Jan 7, 2026
Member
Author
|
Thanks Kostas! |
dnhatn
added
the
auto-backport
dnhatn
added a commit
to dnhatn/elasticsearch
that referenced
this pull request
Jan 7, 2026
For rate aggregations, we buffer raw input data points until all slices are processed. Buffers can be periodically flushed when the slice changes, since each slice contains all data points for a shard within a specific time range. This approach helps reduce memory pressure. However, if the number of slices is smaller than the number of active drivers, it will not help. We should partition the data into multiple reasonably sized slices. I will follow up with this change soon.
Collaborator
💚 Backport successful
|
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jan 7, 2026
For rate aggregations, we buffer raw input data points until all slices are processed. Buffers can be periodically flushed when the slice changes, since each slice contains all data points for a shard within a specific time range. This approach helps reduce memory pressure. However, if the number of slices is smaller than the number of active drivers, it will not help. We should partition the data into multiple reasonably sized slices. I will follow up with this change soon.
szybia
added a commit
to szybia/elasticsearch
that referenced
this pull request
Jan 7, 2026
* upstream/main: Add hook for blocking termination (elastic#133555) Delegate to ES93ScalarQuantizedVectorsFormat rather than copying behaviour (elastic#139834) Refactor compound block types (elastic#140219) Flush the rate buffer when the slice index changes (elastic#138856) Log linked project connection errors at debug during shutdown (elastic#140239) Periodic FIPS 140-3 buildkite pipelines (elastic#139909) ES|QL - Remove TERM function (elastic#139953) Fix name of started time field in shutdown status (elastic#139910) Drop `project_routing` from query params (elastic#140272) Fix flaky test: AllocationDecidersTests (elastic#140271) Add List Reindex API (elastic#140184) Expose _tier metadata attribute in ESQL (elastic#139894) Tweak TSDBRestEsqlIT#testTimeSeriesQuerying(...) (elastic#140210) Fix an OOM error when creating to many chained synonym graph token filter. (elastic#140026) Suppress Azure SDK error logs (elastic#139730) Rewritten integer sorts need to use SortedNumericSortField (elastic#139538) (elastic#139700) Adjust index versions for skippers for time series (elastic#139670) Fix host.name skippers index version range (elastic#139636) Remove BWC shim for a broken commit Fix index.mapping.use_doc_values_skipper defaults in serverless (elastic#139532)
sidosera
pushed a commit
to sidosera/elasticsearch
that referenced
this pull request
Jan 7, 2026
For rate aggregations, we buffer raw input data points until all slices are processed. Buffers can be periodically flushed when the slice changes, since each slice contains all data points for a shard within a specific time range. This approach helps reduce memory pressure. However, if the number of slices is smaller than the number of active drivers, it will not help. We should partition the data into multiple reasonably sized slices. I will follow up with this change soon.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For rate aggregations, we buffer raw input data points until all slices are processed. Buffers can be periodically flushed when the slice changes, since each slice contains all data points for a shard within a specific time range. This approach helps reduce memory pressure. However, if the number of slices is smaller than the number of active drivers, it will not help. We should partition the data into multiple reasonably sized slices. I will follow up with this change soon.