Skip to content

Add telemetry for ignored fields#220872

Closed
maryam-saeidi wants to merge 22 commits intoelastic:mainfrom
maryam-saeidi:add-telemetry-for-ignored-fields
Closed

Add telemetry for ignored fields#220872
maryam-saeidi wants to merge 22 commits intoelastic:mainfrom
maryam-saeidi:add-telemetry-for-ignored-fields

Conversation

@maryam-saeidi
Copy link
Copy Markdown
Member

@maryam-saeidi maryam-saeidi commented May 16, 2025

Closes #220815

Summary

This PR adds telemetry for _ignored fields. The goal is to have an alert based on this field and get notified in case a user hits the mapping limit, but by default, we don't expect this to happen.

This PR adds count_ignored_fields_by_rule_type field that counts the number of _ignored fields per rule type.

In the future, we can extend the telemetry data to also include the actual number of mappings over the limit (there is a feature request for adding field count information to index API)

How to test

  • Add a lot of dynamic fields as mentioned here: Auto increase fields limit of the alert indices #216719
  • Create a rule with a custom threshold rule with multiple group by fields to generate an alert with _ignored field
  • Run the following API and check the value of count_ignored_fields_by_rule_type
    POST kbn:/internal/telemetry/clusters/_stats?apiVersion=2
    {
      "unencrypted": true,
      "refreshCache": true
    }
    
Here is what it looks like:

image

image

@maryam-saeidi maryam-saeidi self-assigned this May 16, 2025
@maryam-saeidi maryam-saeidi added release_note:skip Skip the PR/issue when compiling release notes backport:version Backport to applied version labels v9.1.0 v8.19.0 labels May 16, 2025
@github-actions github-actions Bot added the author:obs-ux-management PRs authored by the obs ux management team label May 16, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

  • Click to trigger kibana-pull-request for this PR!
  • Click to trigger kibana-deploy-project-from-pr for this PR!
  • Click to trigger kibana-deploy-cloud-from-pr for this PR!

@maryam-saeidi
Copy link
Copy Markdown
Member Author

Closing in favor of #221480

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author:obs-ux-management PRs authored by the obs ux management team backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes v8.19.0 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Observability Alerting] Add telemetry for number of mappings for each rule type

2 participants