[ResponseOps][Alerting] Create a task to regenerate maintenance window events by js-jankisalvi · Pull Request #219261 · elastic/kibana

js-jankisalvi · 2025-04-25T15:10:00Z

Summary

Resolves #211534

This PR adds a recurring task which will

run once every day
collect maintenance windows which have expiration date within 1 week
updates expiration date to +1 year if it is recurring
generate events for the next 1 year
adds new events to maintenance window

Checklist

Unit or functional tests were updated or added to match the most common scenarios

How to test

Set expiration date to less than 1 week before creating maintenance windows: update line 70 expirationDate = moment().utc().add(1, 'year').toISOString(); to expirationDate = moment().utc().add(5, 'days').toISOString(); in the file x-pack/platform/plugins/shared/alerting/server/application/maintenance_window/methods/create/create_maintenance_window.ts
Create maintenance windows with different scenarios (recurring, non recurring, etc.)
Update task schedule to run every five minutes to test: set { interval: '1d' } to { interval: '5m' } in file x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts
Verify the task ran successfully
Verify maintenance windows are updated properly with new expiration date and new events

Flaky test runner:

https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8358

elasticmachine · 2025-04-29T10:17:57Z

Pinging @elastic/response-ops (Team:ResponseOps)

js-jankisalvi · 2025-04-29T10:58:38Z

...d/alerting/server/application/maintenance_window/methods/create/create_maintenance_window.ts

+  if (rRule.interval || rRule.freq) {
+    expirationDate = moment().utc().add(1, 'year').toISOString();
+  } else {
+    expirationDate = moment(rRule.dtstart).utc().add(duration, 'ms').toISOString();


if maintenance window has non recurring schedule, we don't need to set expiration date for more +1 year.

…om/js-jankisalvi/kibana into recreate-maintenance-window-events

cnasikas

LGTM! I left some comments.

...red/alerting/server/application/maintenance_window/lib/generate_maintenance_window_events.ts

...d/alerting/server/application/maintenance_window/methods/update/update_maintenance_window.ts

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

cnasikas · 2025-05-05T21:18:27Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

+      return {
+        ...filteredMaintenanceWindow,
+        expirationDate: newEvents.length ? newExpirationDate : oldExpirationDate,
+        events: [...oldEvents, ...newEvents],


Do we need to keep the old events? I am wondering a) if the old events become so big that they cannot fit in memory, and b) if the events array becomes too big and the ES grows too much. The new events can contain some past events to be sure we do not miss anything. We can do that by setting the startDate of the generateMaintenanceWindowEvents to be one week in the past.

startDateRange used in filter is 1 week before expiration date, hence used the same date to generate new events.

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

…-fix'

…t --include-path /api/status --include-path /api/alerting/rule/ --include-path /api/alerting/rules --include-path /api/actions --include-path /api/security/role --include-path /api/spaces --include-path /api/streams --include-path /api/fleet --include-path /api/dashboards --include-path /api/saved_objects/_import --include-path /api/saved_objects/_export --include-path /api/maintenance_window --update'

ymao1

Left a few comments. It would be good to add a functional test for this as well.

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

ymao1 · 2025-06-03T18:44:37Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

+    .map((filteredMaintenanceWindow) => {
+      const { rRule, duration, expirationDate: oldExpirationDate } = filteredMaintenanceWindow;
+
+      const newEvents = generateMaintenanceWindowEvents({


seems like generateMaintenanceWindowEvents can throw an error. should we handle that in a try/catch here? otherwise an error in one maintenance window might cause all not to get updated?

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

adcoelho · 2025-06-06T15:15:40Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

+  const fullQuery = [
+    MaintenanceWindowStatus.Running,
+    MaintenanceWindowStatus.Upcoming,
+    MaintenanceWindowStatus.Finished,


Are we including finished maintenance windows in the query to check which ones need events? why?

yes, suppose there was a maintenance window with 2 years of recurring schedule and runs once a year. So once this year's event is done it will be updated as finished. But it's end date is next year. That means it should generate event for next year even though it is finished as of now.

adcoelho · 2025-06-06T15:25:55Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

+    for await (const findResults of soFinder.find()) {
+      mwsWithNewEvents = await generateEvents({
+        maintenanceWindowsSO: findResults.saved_objects,


How does this work? soFinder.find() can return multiple findResults with their own saved_objects? why doesn't it return a single findResult with everything?

It calls savedObjectsClient.find and iterates over multiple pages. It opens a new Point-In-Time (PIT), and continue paging until a set of results is received that's smaller than the designated perPage size. In our case if it has more than 1000 results, it will return in multiple batches. That's why we loop through findResults to find and update maximum 1000 SOs at a time.

js-jankisalvi · 2025-06-06T17:00:45Z

Left a few comments. It would be good to add a functional test for this as well.

Added a functional test

…-fix'

…dow-events

kibanamachine · 2025-06-10T12:03:24Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#8358

[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config.ts: 25/25 tests passed.
[✅] x-pack/test/alerting_api_integration/security_and_spaces/group3/config_with_schedule_circuit_breaker.ts: 25/25 tests passed.

see run history

…dow-events

cnasikas

LGTM! I left some comments.

...ing_api_integration/security_and_spaces/group3/tests/maintenance_window/events_generation.ts

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

cnasikas · 2025-06-15T16:15:30Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

+            };
+          });
+
+          const result = await savedObjectsClient.bulkUpdate<MaintenanceWindowAttributes>(


Let's log the errors returned by the savedObjectsClient.bulkUpdate. bulkUpdate will not throw an error if any of the SOs failed to be updated. Instead, the error property of the SO in the response will not be empty and will have a message. We should iterate through the results and log the errors as Patrick suggested.

Updated
Also subtracted MW SO with errors from totalUpdatedMaintenanceWindows

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.test.ts

cnasikas · 2025-06-15T16:22:38Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.test.ts

+  references: [],
+};
+
+const statusFilter = {


It is hard to understand the filter and verify if it is correct. What about writing it on KQL and then using the fromKueryExpression to convert it to the format you need?

Done I still added a snapshot test for the whole object.

cnasikas · 2025-06-15T16:24:55Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.test.ts

Thanks for the great test coverage!

cnasikas · 2025-06-15T16:28:34Z

...ing_api_integration/security_and_spaces/group3/tests/maintenance_window/events_generation.ts

+      // verify 2 maintenance windows are updated
+      await retry.try(async () => {
+        const maintenanceWindowsResult = await getUpdatedMaintenanceWindows();
+        expect(maintenanceWindowsResult.length).to.eql(2);


Could you please help me understand why two? In getUpdatedMaintenanceWindows, the number of "should be updated as..." is three, and I got confused.

Two is correct number. Sorry my bad with incorrect comment.
The running MW should not be updated as its expiration date is 10 days in future while our task has 1 week past and 1 week future window.

elasticmachine · 2025-06-16T13:35:14Z

💚 Build Succeeded

Buildkite Build
Commit: b3bf14d

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`@kbn/test-suites-xpack-platform`	340	341	+1

Total ESLint disabled count

id	before	after	diff
`@kbn/test-suites-xpack-platform`	344	345	+1

History

💚 Build #308305 succeeded 01e94e4
💛 Build #306877 was flaky d2be24d
💔 Build #306530 failed c59b0be
💔 Build #306367 failed 70d2639
💛 Build #304729 was flaky c6aee26
💛 Build #303638 was flaky efba636

cc @js-jankisalvi

cnasikas

LGTM! Great work!

cnasikas · 2025-06-17T11:46:54Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.test.ts

  describe('getStatusFilter', () => {
    test('should build status filter', () => {
-      expect(getStatusFilter()).toEqual(statusFilter);
+      expect(getStatusFilter()).toMatchInlineSnapshot(`


super nit: I was thinking something like expect(getStatusFilter()).toEqual(fromKueryExpression(statusFilter))

cnasikas · 2025-06-18T11:15:25Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

+        }
+
+        logger.debug(
+          `Cancelling maintenance windows events generator task - execution error due to timeout.`


super nit: We are not sure why the task got cancelled.

cnasikas · 2025-06-18T11:16:00Z

x-pack/platform/plugins/shared/alerting/server/maintenance_window_events/task.ts

+
+        await soFinder?.close();
+
+        return;


super nit:

Suggested change

return;

kibanamachine · 2025-06-18T11:46:32Z

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15731950402

kibanamachine · 2025-06-18T11:52:44Z

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts You might need to backport the following PRs to 8.19: - [Discover][APM] Add link to view related errors in Discover from the full-screen waterfall (#224033) - 🌊 Streams: Simplify onboarding (#224068) - [Logs UX] Replace enzyme with React Testing Library (#224304) - [EDR Workflows] Add runscript from Microsoft Defender for Endpoint support to Response Console (#222377) - [Console] Add ESQL autocomplete support (#219980) - [ES\|QL] Add query description in the documentation prop (#224195) - [Security Solution] [Elastic Defend] Flip the advanced policy for file origin info collection from Opt-Out to Opt-In (#223882) - UI bug when editing Elasticsearch query alert rule "GROUPED OVER" a runtime_mapping field (#223975) - [Infra] Add retry to url assert (#224126) - [ResponseOps][Alerting] assure grouping fields in alert match expected cardinality (#223409)

Manual backport

To create the backport manually run:

node scripts/backport --pr 219261

Questions ?

Please refer to the Backport tool documentation

kibanamachine · 2025-06-20T11:48:43Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 219261 locally
cc: @js-jankisalvi

cnasikas · 2025-06-21T17:18:33Z

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…e window events (#219261) (#224773) # Backport This will backport the following commits from `main` to `8.19`: - [[ResponseOps][Alerting] Create a task to regenerate maintenance window events (#219261)](#219261)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Janki Salvi <117571355+js-jankisalvi@users.noreply.github.com>

js-jankisalvi added 2 commits April 23, 2025 15:22

initial commit

277ef56

unit tests

5d355a3

js-jankisalvi self-assigned this Apr 25, 2025

js-jankisalvi and others added 3 commits April 25, 2025 16:33

Merge branch 'main' into recreate-maintenance-window-events

6ec0681

don't update expiration date when no new events

a652beb

Merge branch 'main' into recreate-maintenance-window-events

e7c3c4c

js-jankisalvi added Team:ResponseOps Platform ResponseOps team (formerly the Cases and Alerting teams) t// v9.1.0 v8.19.0 Feature:Alerting/RulesFramework Issues related to the Alerting Rules Framework labels Apr 29, 2025

js-jankisalvi changed the title ~~[ResponseOps][Alerting] Generate maintenance window events task~~ [ResponseOps][Alerting] Task to generate maintenance window events Apr 29, 2025

js-jankisalvi added the release_note:skip Skip the PR/issue when compiling release notes label Apr 29, 2025

js-jankisalvi marked this pull request as ready for review April 29, 2025 10:17

js-jankisalvi requested a review from a team as a code owner April 29, 2025 10:17

js-jankisalvi added the backport:version Backport to applied version labels label Apr 29, 2025

js-jankisalvi changed the title ~~[ResponseOps][Alerting] Task to generate maintenance window events~~ [ResponseOps][Alerting] Create a task to generate maintenance window events Apr 29, 2025

js-jankisalvi commented Apr 29, 2025

View reviewed changes

js-jankisalvi and others added 3 commits April 29, 2025 12:38

add to registered task types test

c5333af

Merge branch 'recreate-maintenance-window-events' of https://github.c…

2e804f9

…om/js-jankisalvi/kibana into recreate-maintenance-window-events

Merge branch 'main' into recreate-maintenance-window-events

e07b290

cnasikas reviewed May 5, 2025

View reviewed changes

js-jankisalvi and others added 5 commits May 6, 2025 16:54

feedback addressed 1

1193b7d

Merge branch 'main' into recreate-maintenance-window-events

a3f3a33

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

2ed9c94

…-fix'

Merge branch 'main' into recreate-maintenance-window-events

1bec795

Feedback 2

b039796

js-jankisalvi changed the title ~~[ResponseOps][Alerting] Create a task to generate maintenance window events~~ [ResponseOps][Alerting] Create a task to regenerate maintenance window events May 27, 2025

js-jankisalvi added 2 commits May 27, 2025 11:29

Merge branch 'main' into recreate-maintenance-window-events

e5de478

Merge branch 'main' into recreate-maintenance-window-events

efba636

kibanamachine added 2 commits June 3, 2025 14:26

[CI] Auto-commit changed files from 'make api-docs'

c6aee26

ymao1 reviewed Jun 3, 2025

View reviewed changes

adcoelho reviewed Jun 6, 2025

View reviewed changes

js-jankisalvi and others added 2 commits June 6, 2025 17:55

added functional test

87ab475

Merge branch 'main' into recreate-maintenance-window-events

70d2639

js-jankisalvi and others added 5 commits June 9, 2025 09:56

fix type

9b2d19e

Merge branch 'main' into recreate-maintenance-window-events

7a808d4

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

c59b0be

…-fix'

lint fix for test file

f13a003

Merge remote-tracking branch 'upstream' into recreate-maintenance-win…

d2be24d

…dow-events

Merge remote-tracking branch 'upstream' into recreate-maintenance-win…

01e94e4

…dow-events

cnasikas reviewed Jun 15, 2025

View reviewed changes

js-jankisalvi and others added 2 commits June 16, 2025 12:18

update cancel task and add unit tests

930aa8b

Merge branch 'main' into recreate-maintenance-window-events

b3bf14d

cnasikas approved these changes Jun 18, 2025

View reviewed changes

js-jankisalvi merged commit eaa9c1c into elastic:main Jun 18, 2025
10 checks passed

js-jankisalvi deleted the recreate-maintenance-window-events branch June 18, 2025 11:46

kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Jun 20, 2025

cnasikas mentioned this pull request Jun 21, 2025

[8.19] [ResponseOps][Alerting] Create a task to regenerate maintenance window events (#219261) #224773

Merged

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Jun 23, 2025

Conversation

js-jankisalvi commented Apr 25, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

How to test

Flaky test runner:

Uh oh!

elasticmachine commented Apr 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cnasikas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ymao1 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

js-jankisalvi Jun 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

js-jankisalvi commented Jun 6, 2025

Uh oh!

kibanamachine commented Jun 10, 2025

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#8358

Uh oh!

cnasikas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jun 16, 2025

💚 Build Succeeded

Metrics [docs]

ESLint disabled line counts

js-jankisalvi commented Apr 25, 2025 •

edited by kibanamachine

Loading

js-jankisalvi Jun 6, 2025 •

edited

Loading