[Detections Response] Extended rule usage telemetry#224968
Merged
yctercero merged 29 commits intoelastic:mainfrom Jul 2, 2025
Merged
[Detections Response] Extended rule usage telemetry#224968yctercero merged 29 commits intoelastic:mainfrom
yctercero merged 29 commits intoelastic:mainfrom
Conversation
yctercero
added
release_note:skip
jkelas
reviewed
Jun 25, 2025
Contributor
jkelas
left a comment
jkelas
left a comment
There was a problem hiding this comment.
Hi @yctercero
I reviewed the code, and as such it looks good, similar to my changes in #222370
But us the code grows in some places, I suggested some improvements / refactorings, to make the code more maintainable. Please take a look and let me know what you think.
...utions/security/plugins/security_solution/server/usage/detections/rules/get_initial_usage.ts
Outdated
Show resolved
Hide resolved
| }, | ||
| }, | ||
| }, | ||
| detection_rules: rulesMetricsSchema, |
Contributor
There was a problem hiding this comment.
I really like this refactoring!
|
|
||
| export interface RulesTypeUsage { | ||
| query: FeatureTypeUsage; | ||
| query_custom: FeatureTypeUsage; |
Contributor
There was a problem hiding this comment.
So now query will represent all query rules.
query_custom will represent only custom query rules.
Probably we would need also elastic_customized query rules, representing prebuilt customized rules, no? What do you think @approksiu ?
I am wondering if this structure is not becoming too messy. Maybe instead of all these fields in RuleTypesUsage, which are of single type, RulesTypeUsage could be an array of objects containing a field telling what it represents (prebuilt rule / prebuilt customized rule / custom rule), and a FeatureTypeUsage. What do you think @yctercero
This could look something like:
RulesTypeUsage:
[{ represents: 'query', featureTypeUsage: {...},
{ represents: 'query_custom', featureTypeUsage: {...}
{ represents: 'query_customized', featureTypeUsage: {...},
{ represents: 'threshold', featureTypeUsage: {...},
{ represents: 'threshold_custom', featureTypeUsage: {...}
{ represents: 'threshold_customized', featureTypeUsage: {...},
...
Or RulesTypeUsage could have 3 fields: all, customized, custom. The value of these fields could be an array of objects having fields for rule types.
something like:
RulesTypeUsage {
all: { threshold: {...}, eql: {...}, ...}
customized: { threshold: {...}, eql: {...}, ...}
custom: { threshold: {...}, eql: {...}, ...}
Contributor
Author
There was a problem hiding this comment.
I do think we could definitely restructure it now that it has grown so big. The reason I didn't go for it here was to not break any existing dashboards. I'm not super familiar with what people tend to do with breaking changes in telemetry.
I think it's worth exploring in the 9.x series.
Contributor
|
Pinging @elastic/security-detection-engine (Team:Detection Engine) |
Bamieh
approved these changes
Jun 25, 2025
Contributor
Bamieh
left a comment
Bamieh
left a comment
There was a problem hiding this comment.
telemetry additions (and description updates) LGTM
yctercero
requested review from
a team and
nkhristinin
and removed request for
a team and
nkhristinin
szaffarano
approved these changes
Jun 28, 2025
Contributor
💚 Build Succeeded
Metrics [docs]
History
cc @yctercero |
Contributor
|
Starting backport for target branches: 8.19, 9.1 https://github.com/elastic/kibana/actions/runs/16035004739 |
kibanamachine
added a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 2, 2025
## Summary Addresses elastic/security-team#12856 Product wanted more granular information on custom rule usage. We have information on number of different rule types, total custom and total prebuilt, but there isn't a clear snapshot of the custom rules broken out by rule type. The existing rule type usage includes both prebuilt and custom. ### Sample <img width="342" alt="Screenshot 2025-06-23 at 2 40 59 PM" src="https://github.com/user-attachments/assets/3dcb3fe0-5251-489f-bf3b-d762e57ab597" /> ### How to test You can run `cd x-pack/solutions/security/plugins/security_solution/scripts/endpoint && yarn test:generate` to generate some test data. - Create some rules of different types - Add some exceptions - Add some rules with suppression - Try adding a few prebuilt rules of different rule types - Visit Stack Management > Advanced Settings > Global settings > Usage Collection > Click on the link `cluster data` - Values of `[rule_type]_custom` should only reflect numbers for custom rules, not prebuilt --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit f282c4d)
kibanamachine
added a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 2, 2025
## Summary Addresses elastic/security-team#12856 Product wanted more granular information on custom rule usage. We have information on number of different rule types, total custom and total prebuilt, but there isn't a clear snapshot of the custom rules broken out by rule type. The existing rule type usage includes both prebuilt and custom. ### Sample <img width="342" alt="Screenshot 2025-06-23 at 2 40 59 PM" src="https://github.com/user-attachments/assets/3dcb3fe0-5251-489f-bf3b-d762e57ab597" /> ### How to test You can run `cd x-pack/solutions/security/plugins/security_solution/scripts/endpoint && yarn test:generate` to generate some test data. - Create some rules of different types - Add some exceptions - Add some rules with suppression - Try adding a few prebuilt rules of different rule types - Visit Stack Management > Advanced Settings > Global settings > Usage Collection > Click on the link `cluster data` - Values of `[rule_type]_custom` should only reflect numbers for custom rules, not prebuilt --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit f282c4d)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jul 2, 2025
…226304) # Backport This will backport the following commits from `main` to `9.1`: - [[Detections Response] Extended rule usage telemetry (#224968)](#224968) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Yara Tercero","email":"yctercero@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-07-02T20:21:57Z","message":"[Detections Response] Extended rule usage telemetry (#224968)\n\n## Summary\n\nAddresses https://github.com/elastic/security-team/issues/12856\n\nProduct wanted more granular information on custom rule usage. We have\ninformation on number of different rule types, total custom and total\nprebuilt, but there isn't a clear snapshot of the custom rules broken\nout by rule type. The existing rule type usage includes both prebuilt\nand custom.\n\n### Sample\n\n<img width=\"342\" alt=\"Screenshot 2025-06-23 at 2 40 59 PM\"\nsrc=\"https://github.com/user-attachments/assets/3dcb3fe0-5251-489f-bf3b-d762e57ab597\"\n/>\n\n### How to test\n\nYou can run `cd\nx-pack/solutions/security/plugins/security_solution/scripts/endpoint &&\nyarn test:generate` to generate some test data.\n\n- Create some rules of different types\n- Add some exceptions\n- Add some rules with suppression\n- Try adding a few prebuilt rules of different rule types\n- Visit Stack Management > Advanced Settings > Global settings > Usage\nCollection > Click on the link `cluster data`\n- Values of `[rule_type]_custom` should only reflect numbers for custom\nrules, not prebuilt\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"f282c4d98fac1e8dfc99fcff4d2a930cea1f30df","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Detection Engine","backport:version","v9.1.0","v8.19.0","v9.2.0"],"title":"[Detections Response] Extended rule usage telemetry","number":224968,"url":"https://github.com/elastic/kibana/pull/224968","mergeCommit":{"message":"[Detections Response] Extended rule usage telemetry (#224968)\n\n## Summary\n\nAddresses https://github.com/elastic/security-team/issues/12856\n\nProduct wanted more granular information on custom rule usage. We have\ninformation on number of different rule types, total custom and total\nprebuilt, but there isn't a clear snapshot of the custom rules broken\nout by rule type. The existing rule type usage includes both prebuilt\nand custom.\n\n### Sample\n\n<img width=\"342\" alt=\"Screenshot 2025-06-23 at 2 40 59 PM\"\nsrc=\"https://github.com/user-attachments/assets/3dcb3fe0-5251-489f-bf3b-d762e57ab597\"\n/>\n\n### How to test\n\nYou can run `cd\nx-pack/solutions/security/plugins/security_solution/scripts/endpoint &&\nyarn test:generate` to generate some test data.\n\n- Create some rules of different types\n- Add some exceptions\n- Add some rules with suppression\n- Try adding a few prebuilt rules of different rule types\n- Visit Stack Management > Advanced Settings > Global settings > Usage\nCollection > Click on the link `cluster data`\n- Values of `[rule_type]_custom` should only reflect numbers for custom\nrules, not prebuilt\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"f282c4d98fac1e8dfc99fcff4d2a930cea1f30df"}},"sourceBranch":"main","suggestedTargetBranches":["9.1","8.19"],"targetPullRequestStates":[{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/224968","number":224968,"mergeCommit":{"message":"[Detections Response] Extended rule usage telemetry (#224968)\n\n## Summary\n\nAddresses https://github.com/elastic/security-team/issues/12856\n\nProduct wanted more granular information on custom rule usage. We have\ninformation on number of different rule types, total custom and total\nprebuilt, but there isn't a clear snapshot of the custom rules broken\nout by rule type. The existing rule type usage includes both prebuilt\nand custom.\n\n### Sample\n\n<img width=\"342\" alt=\"Screenshot 2025-06-23 at 2 40 59 PM\"\nsrc=\"https://github.com/user-attachments/assets/3dcb3fe0-5251-489f-bf3b-d762e57ab597\"\n/>\n\n### How to test\n\nYou can run `cd\nx-pack/solutions/security/plugins/security_solution/scripts/endpoint &&\nyarn test:generate` to generate some test data.\n\n- Create some rules of different types\n- Add some exceptions\n- Add some rules with suppression\n- Try adding a few prebuilt rules of different rule types\n- Visit Stack Management > Advanced Settings > Global settings > Usage\nCollection > Click on the link `cluster data`\n- Values of `[rule_type]_custom` should only reflect numbers for custom\nrules, not prebuilt\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"f282c4d98fac1e8dfc99fcff4d2a930cea1f30df"}}]}] BACKPORT--> Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>
kibanamachine
added a commit
that referenced
this pull request
Jul 2, 2025
…226303) # Backport This will backport the following commits from `main` to `8.19`: - [[Detections Response] Extended rule usage telemetry (#224968)](#224968) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Yara Tercero","email":"yctercero@users.noreply.github.com"},"sourceCommit":{"committedDate":"2025-07-02T20:21:57Z","message":"[Detections Response] Extended rule usage telemetry (#224968)\n\n## Summary\n\nAddresses https://github.com/elastic/security-team/issues/12856\n\nProduct wanted more granular information on custom rule usage. We have\ninformation on number of different rule types, total custom and total\nprebuilt, but there isn't a clear snapshot of the custom rules broken\nout by rule type. The existing rule type usage includes both prebuilt\nand custom.\n\n### Sample\n\n<img width=\"342\" alt=\"Screenshot 2025-06-23 at 2 40 59 PM\"\nsrc=\"https://github.com/user-attachments/assets/3dcb3fe0-5251-489f-bf3b-d762e57ab597\"\n/>\n\n### How to test\n\nYou can run `cd\nx-pack/solutions/security/plugins/security_solution/scripts/endpoint &&\nyarn test:generate` to generate some test data.\n\n- Create some rules of different types\n- Add some exceptions\n- Add some rules with suppression\n- Try adding a few prebuilt rules of different rule types\n- Visit Stack Management > Advanced Settings > Global settings > Usage\nCollection > Click on the link `cluster data`\n- Values of `[rule_type]_custom` should only reflect numbers for custom\nrules, not prebuilt\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"f282c4d98fac1e8dfc99fcff4d2a930cea1f30df","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Detection Engine","backport:version","v9.1.0","v8.19.0","v9.2.0"],"title":"[Detections Response] Extended rule usage telemetry","number":224968,"url":"https://github.com/elastic/kibana/pull/224968","mergeCommit":{"message":"[Detections Response] Extended rule usage telemetry (#224968)\n\n## Summary\n\nAddresses https://github.com/elastic/security-team/issues/12856\n\nProduct wanted more granular information on custom rule usage. We have\ninformation on number of different rule types, total custom and total\nprebuilt, but there isn't a clear snapshot of the custom rules broken\nout by rule type. The existing rule type usage includes both prebuilt\nand custom.\n\n### Sample\n\n<img width=\"342\" alt=\"Screenshot 2025-06-23 at 2 40 59 PM\"\nsrc=\"https://github.com/user-attachments/assets/3dcb3fe0-5251-489f-bf3b-d762e57ab597\"\n/>\n\n### How to test\n\nYou can run `cd\nx-pack/solutions/security/plugins/security_solution/scripts/endpoint &&\nyarn test:generate` to generate some test data.\n\n- Create some rules of different types\n- Add some exceptions\n- Add some rules with suppression\n- Try adding a few prebuilt rules of different rule types\n- Visit Stack Management > Advanced Settings > Global settings > Usage\nCollection > Click on the link `cluster data`\n- Values of `[rule_type]_custom` should only reflect numbers for custom\nrules, not prebuilt\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"f282c4d98fac1e8dfc99fcff4d2a930cea1f30df"}},"sourceBranch":"main","suggestedTargetBranches":["9.1","8.19"],"targetPullRequestStates":[{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/224968","number":224968,"mergeCommit":{"message":"[Detections Response] Extended rule usage telemetry (#224968)\n\n## Summary\n\nAddresses https://github.com/elastic/security-team/issues/12856\n\nProduct wanted more granular information on custom rule usage. We have\ninformation on number of different rule types, total custom and total\nprebuilt, but there isn't a clear snapshot of the custom rules broken\nout by rule type. The existing rule type usage includes both prebuilt\nand custom.\n\n### Sample\n\n<img width=\"342\" alt=\"Screenshot 2025-06-23 at 2 40 59 PM\"\nsrc=\"https://github.com/user-attachments/assets/3dcb3fe0-5251-489f-bf3b-d762e57ab597\"\n/>\n\n### How to test\n\nYou can run `cd\nx-pack/solutions/security/plugins/security_solution/scripts/endpoint &&\nyarn test:generate` to generate some test data.\n\n- Create some rules of different types\n- Add some exceptions\n- Add some rules with suppression\n- Try adding a few prebuilt rules of different rule types\n- Visit Stack Management > Advanced Settings > Global settings > Usage\nCollection > Click on the link `cluster data`\n- Values of `[rule_type]_custom` should only reflect numbers for custom\nrules, not prebuilt\n\n---------\n\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"f282c4d98fac1e8dfc99fcff4d2a930cea1f30df"}}]}] BACKPORT--> Co-authored-by: Yara Tercero <yctercero@users.noreply.github.com>
kertal
pushed a commit
to kertal/kibana
that referenced
this pull request
Jul 25, 2025
## Summary Addresses elastic/security-team#12856 Product wanted more granular information on custom rule usage. We have information on number of different rule types, total custom and total prebuilt, but there isn't a clear snapshot of the custom rules broken out by rule type. The existing rule type usage includes both prebuilt and custom. ### Sample <img width="342" alt="Screenshot 2025-06-23 at 2 40 59 PM" src="https://github.com/user-attachments/assets/3dcb3fe0-5251-489f-bf3b-d762e57ab597" /> ### How to test You can run `cd x-pack/solutions/security/plugins/security_solution/scripts/endpoint && yarn test:generate` to generate some test data. - Create some rules of different types - Add some exceptions - Add some rules with suppression - Try adding a few prebuilt rules of different rule types - Visit Stack Management > Advanced Settings > Global settings > Usage Collection > Click on the link `cluster data` - Values of `[rule_type]_custom` should only reflect numbers for custom rules, not prebuilt --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses https://github.com/elastic/security-team/issues/12856
Product wanted more granular information on custom rule usage. We have information on number of different rule types, total custom and total prebuilt, but there isn't a clear snapshot of the custom rules broken out by rule type. The existing rule type usage includes both prebuilt and custom.
Sample
How to test
You can run
cd x-pack/solutions/security/plugins/security_solution/scripts/endpoint && yarn test:generateto generate some test data.cluster data[rule_type]_customshould only reflect numbers for custom rules, not prebuilt