[RAM] Add alert summary API

[XavierM]

Summary

Resolve: #141487

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[simianhacker]

When I include featureIDs I get an empty response:

curl --location --request POST 'http://localhost:5601/internal/rac/alerts/_alert_summary' \
--header 'Content-Type: application/json' \
--header 'kbn-xsrf: true' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
--data-raw '{
    "fixed_interval": "1m",
    "gte": "2022-11-30T16:38:22.710Z",
    "lte": "2022-11-30T17:38:22.710Z",
    "featureIds": [
        "infrastructure"
    ]
}'

But when I omit the featureIDs I get the response I'm expecting

curl --location --request POST 'http://localhost:5601/internal/rac/alerts/_alert_summary' \
--header 'Content-Type: application/json' \
--header 'kbn-xsrf: true' \
--header 'Authorization: Basic ZWxhc3RpYzpjaGFuZ2VtZQ==' \
--data-raw '{
    "fixed_interval": "1m",
    "gte": "2022-11-30T16:38:22.710Z",
    "lte": "2022-11-30T17:38:22.710Z"
}'

I did a little digging and it appears the index it's trying to use with the infrastructure feature ID is .alerts-observability.metrics.alerts but the index actual index name (per Dev Console) is .alerts-observability.metrics.alerts-default.

[simianhacker]

I think the way we are querying the "recovered alerts" is wrong. When an alert recovers the recovery should only show up in the bucket it recovered based on kibana.alert.end instead of using the kibana.alert.time_range. Currently, we are drawing the recovered alerts from when the alert started:

I think we want something like this:

Where 200 alerts were active from 2:12 pm to 2:16 pm and then at 2:16 pm all 200 alerts recovered. Here is how I modified the recovered alerts query for the second chart:

POST .alerts-*/_search
{
  "aggs": {
    "recovered_alerts_bucket": {
      "date_histogram": {
        "field": "kibana.alert.end",
        "fixed_interval": "1m",
        "extended_bounds": {
          "min": "2022-11-30T20:46:52.631Z",
          "max": "2022-11-30T21:46:52.631Z"
        }
      }
    }
  },
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "kibana.alert.end": {
              "gt": "2022-11-30T20:46:52.631Z",
              "lt": "2022-11-30T21:46:52.631Z"
            }
          }
        },
        {
          "term": {
            "kibana.alert.status": "recovered"
          }
        }
      ]
    }
  },
  "size": 0
}

@vinaychandrasekhar @maciejforcone Am I interpreting the design correctly?

[maciejforcone]

I also think that second makes more sense. User needs to see the duration of active alerts (so it should be a timerange to use for further investigation), where for recovered we should use just a moment of recovery (so it's a timestamp, when all went back to normal). And for the chart type, are we moving from line to area? I wanted us to use the same visual pattern that we have in APM, so line = recent timerange, area = previous timerange, for the comparison use case.

[XavierM]

@simianhacker , we do not need the hard_bounds for the recovered alerts, correct? Look what I did and let me know if i need to change it. I also fixed your first observation :)

[watson]

Kibana Platform Security changes LGTM 👍

[maryam-saeidi]

When I was checking the result, the key_as_string was not the same for active and recovered:

Update: After a discussion with @XavierM, I learned that this format is returned from the related query and we can use key in our charts instead. So this difference will be kept as it is.

[maryam-saeidi]

What does this option do?

[XavierM]

This is for kibana security, I think we created a special rac access.

[maryam-saeidi]

Can we use DateFromString instead to validate it at the body validation level?

[XavierM]

I only imagined that will use this UTC format and not just a date. However, if you feel that we should allow just date without time, I can change it back to moment.ISO_8601

[maryam-saeidi]

No, it is fine, I also used the same time format in my PR for the related component.

[maryam-saeidi]

I wasn't able to check the graphs as was shown by @simianhacker, so I'll wait for his approval from our team.

I wasn't able to check the graphs as was shown by @simianhacker, so I'll wait for his approval from our team.

[simianhacker]

LGTM

[ymao1]

Should we validate this with a custom validator on the body params?

[XavierM]

I did not find an easy way to do it with io-ts, so I will keep it this way for now.

[ymao1]

LGTM

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: f892e9c

Failed CI Steps

• FTR Configs #6

Test Failures

• [job] [logs] FTR Configs #6 / endpoint endpoint list when there is data, when the hostname is clicked on, for the kql filtering for united.endpoint.host.hostname, table shows 1 item
• [job] [logs] FTR Configs #6 / endpoint endpoint list when there is data, when the hostname is clicked on, for the kql query: na, table shows an empty list

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
ruleRegistry	211	213	+2

Unknown metric groups

API count

id	before	after	diff
ruleRegistry	239	241	+2

ESLint disabled in files

id	before	after	diff
osquery	1	2	+1

ESLint disabled line counts

id	before	after	diff
enterpriseSearch	19	21	+2
fleet	61	67	+6
osquery	109	115	+6
securitySolution	440	446	+6
total			+20

Total ESLint disabled count

id	before	after	diff
enterpriseSearch	20	22	+2
fleet	70	76	+6
osquery	110	117	+7
securitySolution	517	523	+6
total			+21

History

• 💚 Build #95252 succeeded f030944
• 💔 Build #94552 failed ce4707c
• 💔 Build #94474 failed df20fc2
• 💚 Build #94150 succeeded 12464f2
• 💚 Build #94110 succeeded 65a0536
• 💚 Build #93163 succeeded 6760d3d

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream