[Custom threshold] Save group by information with dynamic mapping

[maryam-saeidi]

Relaetd to #183248
Auto-increasing mapping limit PR: #216719

Summary

In this PR, we are saving dynamically mapped group by information for the custom threshold rule. This consists of two parts:

1. Adding a dynamic field

// kibana.alert.grouping
[ALERT_GROUPING]: {
    type: 'object',
    dynamic: true,
    array: false,
    required: false,
  },

2. Adding a dynamic template

dynamicTemplates: [
      {
        strings_as_keywords: {
          path_match: 'kibana.alert.grouping.*',
          match_mapping_type: 'string',
          mapping: {
            type: 'keyword',
            ignore_above: 1024,
          },
        },
      },
    ],

The result of adding these mappings can be seen below:

Alert	Mapping

If the number of mapping limit is exceeded, the fields that are not mapped are going to be added to the _ignored field, but the value is available in the doc.

[prodsecmachine]

🎉 Snyk checks have passed. No issues have been found so far.

✅ security/snyk check is complete. No issues have been found. (View Details)

✅ license/snyk check is complete. No issues have been found. (View Details)

[elasticmachine]

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

[mgiota]

@maryam-saeidi I read through the above mentioned issues to get the context and understand what I need to review here.

Here's a rule that I created and what I got. I created a rule with group by host.name and tags and I added test in the tags field. Is it ok that I got both kibana.alert.group and kibana.alert.grouping? Notice how the tags is saved.

[maryam-saeidi]

@maryam-saeidi I read through the above mentioned issues to get the context and understand what I need to review here.

Here's a rule that I created and what I got. I created a rule with group by host.name and tags and I added test in the tags field. Is it ok that I got both kibana.alert.group and kibana.alert.grouping? Notice how the tags is saved.

@mgiota Yes, having both kibana.alert.group and kibana.alert.grouping is OK. We don't have a plan to remove kibana.alert.group as we are using it in many places, and users might rely on it, so this PR only adds kibana.alert.grouping as an addition.

Regarding tags, I will take a look to see what the issue is, but since Kevin mentioned this issue before, it seems unrelated to this PR.

[mgiota]

LGTM!

[maryam-saeidi]

@mgiota For the tag issue, I checked locally and it was also happening for the ES Query rule. I've created a ticket for it: #221252

[ersin-erdal]

ResponseOps changes LGTM.
Just a note: legacyExperimentalFieldMap is used in some different places as well, such as sloBurnRateRuleType, apmRuleTypeAlertFieldMap or timelines. So this new field will be added there too.

[maryam-saeidi]

ResponseOps changes LGTM. Just a note: legacyExperimentalFieldMap is used in some different places as well, such as sloBurnRateRuleType, apmRuleTypeAlertFieldMap or timelines. So this new field will be added there too.

We will add this field to other observability rules as well, so it aligns with the direction that we are going.
Thanks for your review 🙏🏻

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 6b77e4f

Failed CI Steps

• FTR Configs #74

Test Failures

• [job] [logs] FTR Configs #74 / Security Solution - Telemetry Security Telemetry - Indices metadata task telemetry @ess indices metadata index stats events should have the expected fields

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/alerting-rule-utils	13	17	+4
@kbn/rule-data-utils	180	181	+1
total			+5

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
apm	2.5MB	2.5MB	+10.0B
cases	1.3MB	1.3MB	+5.0B
embeddableAlertsTable	853.9KB	853.9KB	+5.0B
infra	1.1MB	1.1MB	+14.0B
ml	5.3MB	5.3MB	+5.0B
observability	1.3MB	1.3MB	+55.0B
securitySolution	9.2MB	9.2MB	+5.0B
slo	923.7KB	923.7KB	+5.0B
triggersActionsUi	1.4MB	1.4MB	+5.0B
total			+109.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
observability	93.5KB	93.5KB	+30.0B
timelines	177.6KB	177.7KB	+73.0B
total			+103.0B

Unknown metric groups

API count

id	before	after	diff
@kbn/alerting-rule-utils	13	17	+4
@kbn/rule-data-utils	192	193	+1
total			+5

History

• 💔 Build #301733 failed 81b23f7
• 💚 Build #300674 succeeded 16b5066
• 💔 Build #300582 failed fe89380
• 💔 Build #300473 failed 6da4ea4
• 💛 Build #300353 was flaky c03baf0

cc @maryam-saeidi

[kibanamachine]

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15227438358

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 219826

Questions ?

Please refer to the Backport tool documentation

[maryam-saeidi]

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation