Attribute ES|QL shard search load in Lucene operators#142841
Merged
jimczi merged 18 commits intoelastic:mainfrom Feb 24, 2026
Merged
Attribute ES|QL shard search load in Lucene operators#142841jimczi merged 18 commits intoelastic:mainfrom
jimczi merged 18 commits intoelastic:mainfrom
Conversation
This PR completes ES|QL shard-level search load accounting by attributing shard search load from Lucene source operators during query execution. It introduces per-shard timing and row emission tracking in LuceneOperator, reports shard load deltas through SourceOperator, and integrates the Driver run loop to ensure load is reported on status updates and operator completion. As a follow-up, these shard-level statistics could also be surfaced in the ES|QL status output.
jimczi
added
>feature
:Analytics/Compute Engine
elasticsearchmachine
added
Team:Analytics
Collaborator
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
Collaborator
|
Hi @jimczi, I've created a changelog YAML for you. |
Collaborator
|
Pinging @elastic/es-search-foundations (Team:Search Foundations) |
…o lucene_operator_search_load
…o lucene_operator_search_load
andreidan
approved these changes
Feb 24, 2026
Contributor
andreidan
left a comment
andreidan
left a comment
There was a problem hiding this comment.
LGTM, thanks for adding this Jim
...plugin/esql/compute/src/main/java/org/elasticsearch/compute/lucene/query/LuceneOperator.java
Outdated
Show resolved
Hide resolved
Contributor
Author
|
@elasticmachine run Elasticsearch Serverless Checks |
szybia
added a commit
to szybia/elasticsearch
that referenced
this pull request
Feb 24, 2026
…on-sliced-reindex
* upstream/main:
Mute org.elasticsearch.reindex.management.ReindexManagementClientYamlTestSuiteIT test {yaml=reindex/30_cancel_reindex/Cancel running reindex returns response and GET confirms completed} elastic#142079
[ESQL] Fix async logging consistency and severity (elastic#142401)
more consistent allow_no_indices=false errors (elastic#142373)
Attribute ES|QL shard search load in Lucene operators (elastic#142841)
MetricsInfoOperator refactoring (elastic#142935)
ivancea
added a commit
that referenced
this pull request
Mar 2, 2026
tballison
pushed a commit
to tballison/elasticsearch
that referenced
this pull request
Mar 3, 2026
Fixes elastic#143111 Fixes elastic#143086 Fixes elastic#142986 Introduced in elastic#142841 The `now` variable holds millis, while the `reportSearchLoad()` expects nanos. Propagated `nowSupplier` there instead to pass the actual nanos.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR completes ES|QL shard-level search load accounting by attributing shard search load from Lucene source operators during query execution. It introduces per-shard timing and row emission tracking in LuceneOperator, reports shard load deltas through SourceOperator, and integrates the Driver run loop to ensure load is reported on status updates and operator completion. As a follow-up, these shard-level statistics could also be surfaced in the ES|QL status output.