[9.0] [Security Solution][Detection Engine] adds async ES|QL query (#216667)#218583
Merged
vitaliidm merged 1 commit intoelastic:9.0from Apr 17, 2025
Merged
[9.0] [Security Solution][Detection Engine] adds async ES|QL query (#216667)#218583vitaliidm merged 1 commit intoelastic:9.0from
vitaliidm merged 1 commit intoelastic:9.0from
Conversation
…#216667) ## Summary - addresses elastic/security-team#11116 (list item 2) Introducing async query would allow to overcome ES request timeout for long running rules and queries. Timeout for ES request is [defined in alerting framework](https://github.com/elastic/kibana/blob/8.18/x-pack/platform/plugins/shared/alerting/server/lib/get_es_request_timeout.ts#L21) and is smaller value out of rule execution timeout or default ES request timeout(which is 5m and hardcoded [here](https://github.com/elastic/kibana/blob/8.18/x-pack/platform/plugins/shared/alerting/server/lib/get_rule_task_timeout.ts)). If ES|QL rule performs a single long-running ES query, it can time out after 5m due to this ES request timeout. This value can't be changed, unlike rule execution timeout. It can be overwritten in Kibana config ``` xpack.alerting.rules.run: timeout: '10m' ruleTypeOverrides: - id: 'siem.esqlRule' timeout: '15m' ``` So, we can encounter situations when rule fails execution after 5m due to ES request timeout, despite a fact it configured with longer timeout of 15m By using async query, we can overcome this limitation and can poll async query results until it completes or rule timeouts More details in internal [issue](elastic/sdh-security-team#1224) --------- Co-authored-by: Ryland Herrick <ryalnd@gmail.com> (cherry picked from commit 3d7aac1) # Conflicts: # x-pack/solutions/security/plugins/security_solution/server/lib/detection_engine/rule_preview/api/preview_rules/route.ts # x-pack/solutions/security/plugins/security_solution/server/lib/detection_engine/rule_types/esql/esql.ts
marshallmain
approved these changes
Apr 17, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto9.0:Questions ?
Please refer to the Backport tool documentation