[9.4] [Alerting] Preserve rule type payload across delayed-to-active graduation (#266012)#268402
Merged
Merged
Conversation
…tion (elastic#266012) ## Summary Resolves elastic#259886. Architectural alternative to elastic#265588. When a delayed alert is reactivated by `delayRecoveredFlappingAlerts` (flap-hold) and crosses `alertDelay` on a run where the executor does **not** report it, the alert builder used to dispatch to `buildNewAlert` with an empty payload — producing an active AAD doc with blank rule type fields (e.g. `kibana.alert.reason`). Rather than skip the graduation in that case, this PR makes the framework own the `delayed -> active` transition explicitly so the resulting active doc is always complete. ### Fix (code) - **`buildDelayedAlert`** now stores the full executor payload on the delayed AAD doc. Previously the delayed doc only carried framework fields. Persisting the rule type payload turns each delayed doc into a usable predecessor. - **`buildGraduatedAlert`** is a new builder dedicated to `delayed -> active` transitions. It deep-merges the predecessor delayed doc with the current run's payload (per-field precedence: current wins, predecessor fills gaps), sets `event.action: 'open'` and `kibana.alert.status: active`, and treats the alert as user-visible for the first time (`severity_improving: false`, no `previous_action_group`). - **`AlertBuilder.buildActiveAlerts`** now branches on `trackedActive` vs `trackedDelayed` to dispatch to ongoing / graduated / new respectively, instead of the previous status check on a single tracked alert. The per-field merge means: | Run shape on graduation | `cleanedPayload[K]` | Resulting field | | --- | --- | --- | | Executor reports `K` | present | fresh value (predecessor shadowed) | | Flap-hold reactivation, no executor report | absent | predecessor's value preserved | | Partial report (some `K` reported) | present for some | executor where present, predecessor where absent | This matches the long-standing semantics of `buildOngoingAlert`, just sourced from the delayed predecessor instead of an active one. How to reproduce the issue (on the 6th execution we see an alert without context): Run | pattern | flappingHistory | active | recovered | activeCount | pending recovered | flapping | AAD status -- | -- | -- | -- | -- | -- | -- | -- | -- 1 | a | T | x | | 1 | 0 | FALSE | delayed 2 | a | T,F | x | | 2 | 0 | FALSE | active 3 | - | T,F,T | | x | 0 | - | FALSE | recovered 4 | - | T,F,T,F | | x | 0 | - | FALSE | recovered 5 | a | T,F,T,F,T | x | | 1 | 0 | FALSE | delayed 6 | - | T,F,T,F,T,T | x | | 2 | 1 | TRUE | active (cherry picked from commit 5a5a2d0)
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
Author
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
cc @ersin-erdal |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto9.4:Questions ?
Please refer to the Backport tool documentation