Skip to content

[Security Solution] Add "matched_indices_count" rule execution metric#259938

Merged
maximpn merged 13 commits intoelastic:mainfrom
maximpn:add-matched_indices_count-rule-execution-metric
Apr 2, 2026
Merged

[Security Solution] Add "matched_indices_count" rule execution metric#259938
maximpn merged 13 commits intoelastic:mainfrom
maximpn:add-matched_indices_count-rule-execution-metric

Conversation

@maximpn
Copy link
Copy Markdown
Contributor

@maximpn maximpn commented Mar 27, 2026

Addresses: #255513
Relates to: #257203

Summary

This PR adds matched_indices_count rule executor metric. On top of that it adds matched_indicator_indices_count for Indicator Match rules to cover indicator match indices.

Details

matched_indices_count rule execution metric represents the number of indices matched during rule execution. While obviously matched indices might be calculated in several ways we stick to the logic used in runExecutionValidation() function. It checks whether rule's index patterns has any matching indices to break rule execution when there are 0 matching indices. Under the hood it boils down to a request the ES field caps API.

@maximpn maximpn self-assigned this Mar 27, 2026
@maximpn maximpn added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Feature:Rule Monitoring Security Solution Detection Rule Monitoring area Team:Detection Rule Management Security Detection Rule Management Team v9.4.0 Feature:DE O11y labels Mar 27, 2026
@banderror banderror added release_note:enhancement and removed release_note:skip Skip the PR/issue when compiling release notes labels Mar 27, 2026
@maximpn maximpn force-pushed the add-matched_indices_count-rule-execution-metric branch from 8fc4dbc to 5c24306 Compare March 31, 2026 09:34
@maximpn
Copy link
Copy Markdown
Contributor Author

maximpn commented Mar 31, 2026

/ci

@maximpn maximpn marked this pull request as ready for review March 31, 2026 12:09
@maximpn maximpn requested review from a team as code owners March 31, 2026 12:09
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

Copy link
Copy Markdown
Contributor

@pmuellr pmuellr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ResponseOps changes LGTM

@maximpn maximpn force-pushed the add-matched_indices_count-rule-execution-metric branch from 8b672c8 to 9588c3d Compare March 31, 2026 21:22
@maximpn maximpn force-pushed the add-matched_indices_count-rule-execution-metric branch from 9588c3d to eb94455 Compare March 31, 2026 21:42
@maximpn maximpn force-pushed the add-matched_indices_count-rule-execution-metric branch from 86f4c94 to c639813 Compare April 1, 2026 09:52
@elastic elastic deleted a comment from elasticmachine Apr 1, 2026
Copy link
Copy Markdown
Contributor

@nikitaindik nikitaindik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @maximpn! I skimmed through the changes and tested the PR locally with the new Execution Results table. The metrics were collected properly in my tests.

One minor technicality I found is that when a data view doesn't match any indices, we write 0 matched_indices_count to metrics, but if a data view doesn't exist at all, we don't write anything. Which is technically correct (nothing was queried), but might be confusing, as the end result is practically the same. Anyways, we can discuss it further when we work on handling 0 / null values for metrics later.

Copy link
Copy Markdown
Contributor

@davismcphee davismcphee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data Discovery changes LGTM 👍

@maximpn maximpn merged commit b28d03b into elastic:main Apr 2, 2026
16 checks passed
@maximpn maximpn deleted the add-matched_indices_count-rule-execution-metric branch April 2, 2026 07:29
mbondyra added a commit to mbondyra/kibana that referenced this pull request Apr 2, 2026
…heck

* commit 'af66aadafa7470ca8ba3e3edd3793bde81fa4596': (31 commits)
  [Scout] Update test config manifests (elastic#260850)
  [SLO]: register alerts schema embeddable (elastic#256570)
  [Discover][Flyout] Update overview fields table with new prop headerVisibility set to false (elastic#260692)
  [AiButton/Security] Migrate ai-related buttons to use custom styles (elastic#259847)
  [One Workflow] Fix connector step icons falling back to generic plugs in YAML editor (elastic#260785)
  [Agent Builder] Dashboard skill: Guard against editing non-ESQL based panels (elastic#260714)
  Security quality gate Cypress cleanup - Periodic Pipeline (elastic#260820)
  [Search] Deprecate search indices in favour of index management (elastic#260210)
  Upgrade dependency @elastic/charts to v71.4.0 (elastic#260593)
  [Security Solution] [HDQ]: integration-based targeting and descriptor versioning (elastic#258418)
  docs(saved-objects): consolidate docs and document scoped vs system client (elastic#260743)
  Fix observability UIAM config and add CPS observability variant (elastic#260485)
  [Security Solution] Add "matched_indices_count" rule execution metric (elastic#259938)
  [SigEvents] Add callout with working promote action. (elastic#260433)
  [Alerting V2] Episode table actions (elastic#260195)
  [Automatic Migration] Add ability to skip Reference Set step in QRadar upload workflow (elastic#259959)
  [Rules] KQL-to-DSL conversion without data view produces incorrect queries for keyword fields for Metric threshold rule (elastic#260046)
  Update dependency lightningcss to v1.32.0 (main) (elastic#259017)
  Update postcss (main) (elastic#255420)
  Migrate server-side apm.addLabels to OTel dual-write helpers (elastic#259619)
  ...
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Apr 2, 2026
…elastic#259938)

**Addresses:** elastic#255513
**Relates to:** elastic#257203

## Summary

This PR adds `matched_indices_count` rule executor metric. On top of that it adds `matched_indicator_indices_count` for Indicator Match rules to cover indicator match indices.

## Details

`matched_indices_count` rule execution metric represents the number of indices matched during rule execution. While obviously matched indices might be calculated in several ways we stick to the logic used in `runExecutionValidation()` function. It checks whether rule's index patterns has any matching indices to break rule execution when there are 0 matching indices. Under the hood it boils down to a request the [ES field caps API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-field-caps).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting Feature:DE O11y Feature:Rule Monitoring Security Solution Detection Rule Monitoring area release_note:enhancement Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants