Handle is_partial in ES|QL query responses in Elasticsearch query rule by ersin-erdal · Pull Request #228118 · elastic/kibana

ersin-erdal · 2025-07-16T01:56:52Z

Resolves: #209408

This PR adds a handler to check is_partial flag returned by the ES|QL query requests.

When an ES|QL query returns is_partial: true, we set the rule status to warning, which shows a warning on the UI. See the below screenshot.

To verify:

Change the line 85 const isPartial = response.is_partial ?? false; in this PR to const isPartial = true to mimic is_partial: true response.

Create an Elasticsearch query rule with ES|QL, make it return some results.
I used the below query to create test data.

POST /test-index/_bulk?refresh
{"index":{}}
{"@timestamp":"2025-07-15T00:30:17-05:00","message":"40.135.0.0 - - [30/Apr/2020:14:30:17 -0500] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"}
{"index":{}}
{"@timestamp":"2025-07-15T01:30:53-05:00","message":"232.0.0.0 - - [30/Apr/2020:14:30:53 -0500] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"}

Then test the above cases, by making the rule return some or zero results (Set the time window to 5 seconds or 5 days), and by changing the group options:
Create an alert if matches are found and Create an alert for each row

elasticmachine · 2025-07-16T16:52:53Z

Pinging @elastic/response-ops (Team:ResponseOps)

…no-cache --fix'

pmuellr · 2025-07-17T18:07:09Z

if Create an alert if matches are found is selected and no records returned, it fails the rule

if Create an alert if matches are found is selected and some records returned, it sets rule status to warning, which shows a warning on the UI. See the below screenshot.

Is there a precedent for failing the rule if no records are returned - like are we doing something similar somewhere else in Kibana? It seems slightly odd, maybe confusing, that in these cases, it either fails or succeeds with a warning, depending on how data came back (none or anything)

ymao1 · 2025-07-17T18:36:31Z

Is there a precedent for failing the rule if no records are returned - like are we doing something similar somewhere else in Kibana? It seems slightly odd, maybe confusing, that in these cases, it either fails or succeeds with a warning, depending on how data came back (none or anything)

Yea I think for the DSL and KQL rule types, we show the partial results warning even if there are no hits returned and don't throw

ymao1 · 2025-07-17T18:37:54Z

Must be based on @mikecote's comments from the issue:

#209408 (comment)

@mikecote Is it a requirement to be failing the rule run for partial empty results? Should we be doing it for KQL and DSL as well?

mikecote · 2025-07-18T16:09:21Z

@mikecote Is it a requirement to be failing the rule run for partial empty results? Should we be doing it for KQL and DSL as well?

@ymao1 Not a requirement, it was just a thought. Warning is perfectly fine and we can revisit if ever it's not sufficient. Consistency makes sense as well.

…ana into 209408-handle-is-partial

ersin-erdal · 2025-07-18T18:59:25Z

I removed the error throwing part, just sets the status to warning.

ymao1

PR works as described. Wondering if there's a way to get a more detailed warning message? It looks like the detection ES|QL rule pulls a message from the response.clusters object if it exists: #223198

…ana into 209408-handle-is-partial

…no-cache --fix'

ersin-erdal · 2025-07-21T19:42:40Z

PR works as described. Wondering if there's a way to get a more detailed warning message? It looks like the detection ES|QL rule pulls a message from the response.clusters object if it exists: #223198

Added the same details here too. I didn't add a check for the failure messages since getting is_partial is enough to show the message and it fallbacks to empty array.

ymao1

LGTM. Left one nit about the warning wording.

ymao1 · 2025-07-22T18:19:46Z

x-pack/platform/plugins/shared/stack_alerts/server/rule_types/es_query/lib/fetch_esql_query.ts

+  return i18n.translate('xpack.stackAlerts.esQuery.partialResultsWarning', {
+    defaultMessage:
+      'The query returned partial results. Some clusters may have been skipped due to timeouts or other issues. Failures: {failures}',
+    values: { failures: JSON.stringify(shardFailures) },


since failures can be an empty array, it would be nice to optionally show Failures: in the message since then the warning will look incomplete. So if we can't extract any useful message from the details, it will just show The query returned partial results. Some clusters may have been skipped due to timeouts or other issues.

…ana into 209408-handle-is-partial

elasticmachine · 2025-07-23T10:51:28Z

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`stackAlerts`	27	29	+2

Unknown metric groups

API count

id	before	after	diff
`stackAlerts`	27	29	+2

History

💔 Build #322114 failed fd61f89
💚 Build #321527 succeeded 34e8381
💛 Build #321370 was flaky 8de3e37
💚 Build #320995 succeeded d2c3d5e
💛 Build #320553 was flaky abd382b

@timestamp

elastic#228118) Resolves: elastic#209408 This PR adds a handler to check `is_partial` flag returned by the ES|QL query requests. When an ES|QL query returns `is_partial: true`, we set the rule status to warning, which shows a warning on the UI. See the below screenshot. ## To verify: Change the [line 85 ](https://github.com/elastic/kibana/pull/228118/files#diff-9b17d68f85f149658c205635c0034104276c12506390ca823ec281323240e390R85)`const isPartial = response.is_partial ?? false;` in this PR to `const isPartial = true` to mimic `is_partial: true` response. Create an Elasticsearch query rule with ES|QL, make it return some results. I used the below query to create test data. ``` POST /test-index/_bulk?refresh {"index":{}} {"@timestamp":"2025-07-15T00:30:17-05:00","message":"40.135.0.0 - - [30/Apr/2020:14:30:17 -0500] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"} {"index":{}} {"@timestamp":"2025-07-15T01:30:53-05:00","message":"232.0.0.0 - - [30/Apr/2020:14:30:53 -0500] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"} ``` Then test the above cases, by making the rule return some or zero results (`Set the time window` to 5 seconds or 5 days), and by changing the group options: `Create an alert if matches are found` and `Create an alert for each row` <img width="996" height="629" alt="Screenshot 2025-07-16 at 17 48 05" src="https://github.com/user-attachments/assets/a131e06d-0710-49ca-b1d3-efd9234de27c" /> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

@timestamp

elastic#228118) Resolves: elastic#209408 This PR adds a handler to check `is_partial` flag returned by the ES|QL query requests. When an ES|QL query returns `is_partial: true`, we set the rule status to warning, which shows a warning on the UI. See the below screenshot. ## To verify: Change the [line 85 ](https://github.com/elastic/kibana/pull/228118/files#diff-9b17d68f85f149658c205635c0034104276c12506390ca823ec281323240e390R85)`const isPartial = response.is_partial ?? false;` in this PR to `const isPartial = true` to mimic `is_partial: true` response. Create an Elasticsearch query rule with ES|QL, make it return some results. I used the below query to create test data. ``` POST /test-index/_bulk?refresh {"index":{}} {"@timestamp":"2025-07-15T00:30:17-05:00","message":"40.135.0.0 - - [30/Apr/2020:14:30:17 -0500] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"} {"index":{}} {"@timestamp":"2025-07-15T01:30:53-05:00","message":"232.0.0.0 - - [30/Apr/2020:14:30:53 -0500] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"} ``` Then test the above cases, by making the rule return some or zero results (`Set the time window` to 5 seconds or 5 days), and by changing the group options: `Create an alert if matches are found` and `Create an alert for each row` <img width="996" height="629" alt="Screenshot 2025-07-16 at 17 48 05" src="https://github.com/user-attachments/assets/a131e06d-0710-49ca-b1d3-efd9234de27c" /> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

Handle is_partial in ES|QL query responses in Elasticsearch query rule

8a80259

ersin-erdal added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// labels Jul 16, 2025

ersin-erdal marked this pull request as ready for review July 16, 2025 16:52

ersin-erdal requested a review from a team as a code owner July 16, 2025 16:52

ersin-erdal and others added 3 commits July 17, 2025 17:09

use i18

ffd2732

Merge branch 'main' into 209408-handle-is-partial

48e78f6

[CI] Auto-commit changed files from 'node scripts/eslint_all_files --…

abd382b

…no-cache --fix'

ersin-erdal added 3 commits July 18, 2025 20:55

Remove throwing error

83df48f

Merge branch '209408-handle-is-partial' of github.com:ersin-erdal/kib…

3604717

…ana into 209408-handle-is-partial

fix test name

d2c3d5e

Merge branch 'main' into 209408-handle-is-partial

8de3e37

ymao1 added the ci:cloud-deploy Create or update a Cloud deployment label Jul 21, 2025

ymao1 reviewed Jul 21, 2025

View reviewed changes

ersin-erdal and others added 3 commits July 21, 2025 18:53

Improve the warning message

93e48c8

Merge branch '209408-handle-is-partial' of github.com:ersin-erdal/kib…

4ca3da0

…ana into 209408-handle-is-partial

[CI] Auto-commit changed files from 'node scripts/eslint_all_files --…

34e8381

…no-cache --fix'

ymao1 approved these changes Jul 22, 2025

View reviewed changes

ersin-erdal and others added 3 commits July 23, 2025 00:58

Fix the warning message

d3a2bb9

Merge branch '209408-handle-is-partial' of github.com:ersin-erdal/kib…

fd61f89

…ana into 209408-handle-is-partial

Merge branch 'main' into 209408-handle-is-partial

e7e72ff

ersin-erdal merged commit 0b7c71b into elastic:main Jul 23, 2025
12 checks passed

kibanamachine added the v9.2.0 label Jul 23, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Handle is_partial in ES|QL query responses in Elasticsearch query rule#228118

Handle is_partial in ES|QL query responses in Elasticsearch query rule#228118
ersin-erdal merged 14 commits intoelastic:mainfrom
ersin-erdal:209408-handle-is-partial

ersin-erdal commented Jul 16, 2025 •

edited

Loading

Uh oh!

elasticmachine commented Jul 16, 2025

Uh oh!

pmuellr commented Jul 17, 2025

Uh oh!

ymao1 commented Jul 17, 2025

Uh oh!

ymao1 commented Jul 17, 2025

Uh oh!

mikecote commented Jul 18, 2025 •

edited

Loading

Uh oh!

ersin-erdal commented Jul 18, 2025

Uh oh!

ymao1 left a comment

Uh oh!

ersin-erdal commented Jul 21, 2025

Uh oh!

ymao1 left a comment

Uh oh!

ymao1 Jul 22, 2025

Uh oh!

elasticmachine commented Jul 23, 2025 •

edited

Loading

API count

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Comments

Conversation

ersin-erdal commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

To verify:

Uh oh!

elasticmachine commented Jul 16, 2025

Uh oh!

pmuellr commented Jul 17, 2025

Uh oh!

ymao1 commented Jul 17, 2025

Uh oh!

ymao1 commented Jul 17, 2025

Uh oh!

mikecote commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ersin-erdal commented Jul 18, 2025

Uh oh!

ymao1 left a comment

Choose a reason for hiding this comment

Uh oh!

ersin-erdal commented Jul 21, 2025

Uh oh!

ymao1 left a comment

Choose a reason for hiding this comment

Uh oh!

ymao1 Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

API count

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ersin-erdal commented Jul 16, 2025 •

edited

Loading

mikecote commented Jul 18, 2025 •

edited

Loading

elasticmachine commented Jul 23, 2025 •

edited

Loading