Expose frozen indices information on the rule health endpoint

[denar50]

Summary

This is a follow up PR to expose the metric frozen_indices_queried_max_count on the rule healthcheck endpoint.
This metric is an aggregation of the metric frozen_indices_queried_count which is calculated upon rule execution. Refer to this PR to see more details about it.

How to test this?

• Run Elastic locally with these additional parameters in order to enable the frozen data tier: -E path.repo="/tmp" -E xpack.searchable.snapshot.shared_cache.size=20GB.
• Use this tutorial to create the snapshot repository and an ILM policy. You can disable rollover for the ILM policy and configure indices to be moved to frozen after 0 days.
• Create an index manually and populate it with a couple of documents.
• Assign the ILM policy to the index you created in the previous step and wait for it to be rolled to frozen. You can run this command to speed up the process:

PUT /_cluster/settings
{
  "persistent": {
    "indices.lifecycle.poll_interval": "10s"
  }
}

You can confirm that the index is indeed in frozen by calling

GET <YOUR_IDX_HERE>/_ilm/explain

phase should be frozen and step should be complete.

• Create a rule querying the frozen index.
• Call the rule health endpoint with:

curl -X POST --user elastic:changeme "http://localhost:5601/internal/detection_engine/health/_rule?date_start=2025-04-29T09:07:39.489Z&date_end=2025-05-01T09:08:39.489Z" \
  -H "Content-Type: application/json" \
  -H "elastic-api-version: 1" \
  -H 'kbn-xsrf: 123' \
  -H "x-elastic-internal-origin: Kibana" \
  --data '{"rule_id":"2f9780b5-7819-4685-ab8e-d817d3701d10"}'

You should see frozen_indices_queried_max_count populated with 1.

[elasticmachine]

Pinging @elastic/security-detection-engine (Team:Detection Engine)

[jkelas]

The code looks fine, but I discovered some issue when testing. I couldn't make the frozen_indices_queried_max_count display 1, it was always set to 0. I paired up with the author, @denar50 , and we worked together on the code, the author needs to get back to testing in his own environment, I passed him all the information / how I set up my environment. Waiting for an update from @denar50.

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: f7610d9

Failed CI Steps

• FTR Configs #47
• FTR Configs #126
• FTR Configs #127

Metrics [docs]

✅ unchanged

History

• 💛 Build #298707 was flaky 6543310
• 💚 Build #298428 succeeded ed21961
• 💚 Build #297613 succeeded b25b71b
• 💔 Build #297530 failed 510cf09

[jkelas]

After pairing up again with the reviewer, we concluded that the reason why it didn't work the previous time was an incorrect time range for the query. After fixing the settings, the behavior is correct, it shows the "frozen_indices_queried_max_count": 1 as expected.

The code looks OK.

Testing was done according to the instruction in the ticket.
My curl command looked like this:

curl -X POST --user elastic:changeme "http://localhost:5621/kbn/internal/detection_engine/health/_rule?date_start=2025-04-29T09:07:39.489Z&date_end=2025-05-09T09:08:39.489Z" \
  -H "Content-Type: application/json" \
  -H "elastic-api-version: 1" \
  -H 'kbn-xsrf: 123' \
  -H "x-elastic-internal-origin: Kibana" \
  --data '{"rule_id":"7ca6fb8f-3f5f-4fea-a577-72eccc8e001c"}' | jq

and at the end of the printout, in the last bucket, I can see this:

        "indexing_duration_ms": {
          "percentiles": {
            "50.0": 4,
            "95.0": 4,
            "99.0": 5.519999999999996,
            "99.9": 5.952000000000005
          }
        },
        "frozen_indices_queried_max_count": 1

[kibanamachine]

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/14909813866

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 219703

Questions ?

Please refer to the Backport tool documentation

[denar50]

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation