[9.1] [AI4DSOC] Change the logic to query the last alert ingested in an integration (#228602)#229723
Merged
kibanamachine merged 1 commit intoelastic:9.1from Jul 28, 2025
Merged
Conversation
…egration (elastic#228602) ## Summary This PR changes the logic used in the AI4DSOC alert summary page integration section to display the last sync value for each integration card displayed at the top of the page In the previous logic we were fetching all dataStreams and use the `last_activity_ms` value to display as the last sync for the integration. We [realized](elastic#220617) that some dataStreams have event not related to ingested alert. For example it could be event mentioning that an error is happening... This meant that the alert summary UI was showing an incorrect last sync value. The new logic leverage a call using ESQL that works as follow: ``` FROM logs-integrationName.alert-default | WHERE event.kind == "alert" | SORT event.ingested DESC | KEEP event.ingested | LIMIT 1 ``` We fetch the information for each integration. We retrieve the last document with `event.kind == 'alert'` then we sort to get the most recent first, select only the `event.ingested` field/value pair and keep only the first document. The value from the `event.ingested` field is then passed to the integration card and displayed for the last sync. The PR also sets up a 30 second interval, to make sure that the values in the integration card last sync section are updated periodically. This will give the user an indication that the rest of the page below (charts and table) might be out of date. https://github.com/user-attachments/assets/b3524777-f994-40dd-90d1-0c757d1ec892 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios elastic#220617 (cherry picked from commit 5729233)
1 task
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
PhilippeOberti
added a commit
that referenced
this pull request
Aug 1, 2025
… an integration (#228602) (#229767) # Backport This will backport the following commits from `main` to `8.19`: - [[AI4DSOC] Change the logic to query the last alert ingested in an integration (#228602)](#228602) <!--- Backport version: 10.0.1 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Philippe Oberti","email":"philippe.oberti@elastic.co"},"sourceCommit":{"committedDate":"2025-07-28T22:05:38Z","message":"[AI4DSOC] Change the logic to query the last alert ingested in an integration (#228602)\n\n## Summary\n\nThis PR changes the logic used in the AI4DSOC alert summary page\nintegration section to display the last sync value for each integration\ncard displayed at the top of the page\n\nIn the previous logic we were fetching all dataStreams and use the\n`last_activity_ms` value to display as the last sync for the\nintegration. We\n[realized](#220617) that some\ndataStreams have event not related to ingested alert. For example it\ncould be event mentioning that an error is happening... This meant that\nthe alert summary UI was showing an incorrect last sync value.\n\nThe new logic leverage a call using ESQL that works as follow:\n```\nFROM logs-integrationName.alert-default\n | WHERE event.kind == \"alert\"\n | SORT event.ingested DESC\n | KEEP event.ingested\n | LIMIT 1\n```\nWe fetch the information for each integration. We retrieve the last\ndocument with `event.kind == 'alert'` then we sort to get the most\nrecent first, select only the `event.ingested` field/value pair and keep\nonly the first document.\nThe value from the `event.ingested` field is then passed to the\nintegration card and displayed for the last sync.\n\nThe PR also sets up a 30 second interval, to make sure that the values\nin the integration card last sync section are updated periodically. This\nwill give the user an indication that the rest of the page below (charts\nand table) might be out of date.\n\n\nhttps://github.com/user-attachments/assets/b3524777-f994-40dd-90d1-0c757d1ec892\n\n### Checklist\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n\nhttps://github.com//issues/220617","sha":"57292335e3cb4011c71e62afa7b3aaea4c5ce034","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Threat Hunting:Investigations","backport:version","v9.1.0","v8.19.0","v9.2.0"],"title":"[AI4DSOC] Change the logic to query the last alert ingested in an integration","number":228602,"url":"https://github.com/elastic/kibana/pull/228602","mergeCommit":{"message":"[AI4DSOC] Change the logic to query the last alert ingested in an integration (#228602)\n\n## Summary\n\nThis PR changes the logic used in the AI4DSOC alert summary page\nintegration section to display the last sync value for each integration\ncard displayed at the top of the page\n\nIn the previous logic we were fetching all dataStreams and use the\n`last_activity_ms` value to display as the last sync for the\nintegration. We\n[realized](#220617) that some\ndataStreams have event not related to ingested alert. For example it\ncould be event mentioning that an error is happening... This meant that\nthe alert summary UI was showing an incorrect last sync value.\n\nThe new logic leverage a call using ESQL that works as follow:\n```\nFROM logs-integrationName.alert-default\n | WHERE event.kind == \"alert\"\n | SORT event.ingested DESC\n | KEEP event.ingested\n | LIMIT 1\n```\nWe fetch the information for each integration. We retrieve the last\ndocument with `event.kind == 'alert'` then we sort to get the most\nrecent first, select only the `event.ingested` field/value pair and keep\nonly the first document.\nThe value from the `event.ingested` field is then passed to the\nintegration card and displayed for the last sync.\n\nThe PR also sets up a 30 second interval, to make sure that the values\nin the integration card last sync section are updated periodically. This\nwill give the user an indication that the rest of the page below (charts\nand table) might be out of date.\n\n\nhttps://github.com/user-attachments/assets/b3524777-f994-40dd-90d1-0c757d1ec892\n\n### Checklist\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n\nhttps://github.com//issues/220617","sha":"57292335e3cb4011c71e62afa7b3aaea4c5ce034"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/229723","number":229723,"state":"MERGED","mergeCommit":{"sha":"1526f93c70e561b07bcb1a081b3f45e73bea36b6","message":"[9.1] [AI4DSOC] Change the logic to query the last alert ingested in an integration (#228602) (#229723)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.1`:\n- [[AI4DSOC] Change the logic to query the last alert ingested in an\nintegration (#228602)](https://github.com/elastic/kibana/pull/228602)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Philippe Oberti <philippe.oberti@elastic.co>"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/228602","number":228602,"mergeCommit":{"message":"[AI4DSOC] Change the logic to query the last alert ingested in an integration (#228602)\n\n## Summary\n\nThis PR changes the logic used in the AI4DSOC alert summary page\nintegration section to display the last sync value for each integration\ncard displayed at the top of the page\n\nIn the previous logic we were fetching all dataStreams and use the\n`last_activity_ms` value to display as the last sync for the\nintegration. We\n[realized](#220617) that some\ndataStreams have event not related to ingested alert. For example it\ncould be event mentioning that an error is happening... This meant that\nthe alert summary UI was showing an incorrect last sync value.\n\nThe new logic leverage a call using ESQL that works as follow:\n```\nFROM logs-integrationName.alert-default\n | WHERE event.kind == \"alert\"\n | SORT event.ingested DESC\n | KEEP event.ingested\n | LIMIT 1\n```\nWe fetch the information for each integration. We retrieve the last\ndocument with `event.kind == 'alert'` then we sort to get the most\nrecent first, select only the `event.ingested` field/value pair and keep\nonly the first document.\nThe value from the `event.ingested` field is then passed to the\nintegration card and displayed for the last sync.\n\nThe PR also sets up a 30 second interval, to make sure that the values\nin the integration card last sync section are updated periodically. This\nwill give the user an indication that the rest of the page below (charts\nand table) might be out of date.\n\n\nhttps://github.com/user-attachments/assets/b3524777-f994-40dd-90d1-0c757d1ec892\n\n### Checklist\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n\nhttps://github.com//issues/220617","sha":"57292335e3cb4011c71e62afa7b3aaea4c5ce034"}}]}] BACKPORT-->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto9.1:Questions ?
Please refer to the Backport tool documentation