[Synthetics] Migrate synthetics alerting API tests to deployment agnostic solution by cesco-f · Pull Request #221139 · elastic/kibana

cesco-f · 2025-05-21T15:24:26Z

This PR closes #212954.

Synthetics alerting API tests have been migrated to deployment agnostic solution.

Serverless

node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts

node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="Synthetics Alerting"

Stateful

node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts

node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="Synthetics Alerting"

wayneseymour

Reviewed files owned by my team:

x-pack/test/api_integration/deployment_agnostic/default_configs/serverless.config.base.ts
x-pack/test/api_integration/deployment_agnostic/default_configs/stateful.config.base.ts
x-pack/test/api_integration/deployment_agnostic/services/alerting_api.ts

CR only, LGTM, as long as the tests pass, specifically I'm curious about the bool filter added to the alerting api. But again, if the tests pass, I'm good with it.

cesco-f · 2025-05-22T13:15:53Z

Reviewed files owned by my team:

x-pack/test/api_integration/deployment_agnostic/default_configs/serverless.config.base.ts

x-pack/test/api_integration/deployment_agnostic/default_configs/stateful.config.base.ts

x-pack/test/api_integration/deployment_agnostic/services/alerting_api.ts

CR only, LGTM, as long as the tests pass, specifically I'm curious about the bool filter added to the alerting api. But again, if the tests pass, I'm good with it.

In the current implementation of waitForAlertInIndex in the alerting api we can only query the alert by rule id.
There are some tests in Synthetics that are passing additional filters, for example to retrieve only recovered alerts. @wayneseymour

elasticmachine · 2025-05-23T15:46:48Z

💚 Build Succeeded

Buildkite Build
Commit: e6a28bb

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`@kbn/test-suites-xpack`	709	707	-2

Total ESLint disabled count

id	before	after	diff
`@kbn/test-suites-xpack`	731	729	-2

History

github-actions · 2025-05-26T13:21:10Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

…t agnostic

elasticmachine · 2025-05-29T09:18:22Z

💚 Build Succeeded

Buildkite Build
Commit: b13f203
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-221139-b13f203884ec

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`@kbn/test-suites-xpack`	709	707	-2

Total ESLint disabled count

id	before	after	diff
`@kbn/test-suites-xpack`	731	729	-2

History

💛 Build #302585 was flaky dc8d196
💛 Build #302391 was flaky ad30646
💚 Build #302251 succeeded 9e23dd7
💚 Build #302073 succeeded e6a28bb
💔 Build #301983 failed a1db36e

mgiota · 2025-05-29T10:01:20Z

...gration/deployment_agnostic/apis/observability/alerting/synthetics/synthetics_rule_helper.ts

-      logger: this.logger,
-      indexName: '.internal.alerts-observability.uptime.alerts-default*',
-      retryDelay: 1000,
+      indexName: '.alerts-observability.uptime.alerts-default*',


Is there any reason why .internal.alerts-observability.uptime.alerts-default was changed to '.alerts-observability.uptime.alerts-default*'

Apparently in stateful the index is .internal.alerts-observability.uptime.alerts-default while in serverless is .ds-.alerts-observability.uptime.alerts-default, so I'm using the alias as suggested by @kdelemme and it works for both environments.

Thanks for explanation!

mgiota

@cesco-f Great work! I ran tests locally for both serverless and stateful configurations and everything works fine!

Serverless

Stateful

mgiota · 2025-05-29T10:08:16Z

I'm curious about the bool filter added to the alerting api

I had similar question, but as long as tests are happy with it, I am happy as well! And yep it makes sense to change it like this, since Synthetics passes additional filters

mgiota · 2025-05-29T10:10:03Z

...integration/deployment_agnostic/apis/observability/alerting/synthetics/custom_status_rule.ts

          ],
        });
-        expect(recoveredResponse.hits.hits[1]._source).property(
+        expect(recoveredResponse.hits.hits[0]._source).property(


Wondering why hits[1] was changed to hits[0]

This test was flaky because we were fetching all the alerts and assuming that the recovered alert was in position 1. But this was not always the case (no idea why) and sometimes it was failing.

So I changed the test to fetch only the recovered alerts, I'm expecting to get only 1 result and so that's why it was changed to hits[0].

Ok makes sense! Good job!

kibanamachine · 2025-05-29T10:19:08Z

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15321651132

kibanamachine · 2025-05-29T10:25:19Z

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts You might need to backport the following PRs to 8.19: - [Deployment Agnostic] Adding registryUrl configuration for DA tests (#221628)

Manual backport

To create the backport manually run:

node scripts/backport --pr 221139

Questions ?

Please refer to the Backport tool documentation

cesco-f · 2025-05-29T10:44:24Z

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…stic solution (elastic#221139) This PR closes elastic#212954. Synthetics alerting API tests have been migrated to deployment agnostic solution. ### Serverless ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="Synthetics Alerting" ``` ### Stateful ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="Synthetics Alerting" ```

…nt agnostic solution (#221139) (#221908) # Backport This will backport the following commits from `main` to `8.19`: - [[Synthetics] Migrate synthetics alerting API tests to deployment agnostic solution (#221139)](#221139)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)

Relocating tests added in #221139 into separate feature flag configs, that are not run on MKI but only Kibana CI. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…stic solution (elastic#221139) This PR closes elastic#212954. Synthetics alerting API tests have been migrated to deployment agnostic solution. ### Serverless ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="Synthetics Alerting" ``` ### Stateful ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="Synthetics Alerting" ```

…c#222371) Relocating tests added in elastic#221139 into separate feature flag configs, that are not run on MKI but only Kibana CI. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…stic solution (elastic#221139) This PR closes elastic#212954. Synthetics alerting API tests have been migrated to deployment agnostic solution. ### Serverless ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/serverless/oblt.serverless.config.ts --grep="Synthetics Alerting" ``` ### Stateful ``` node scripts/functional_tests_server --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts node scripts/functional_test_runner --config x-pack/test/api_integration/deployment_agnostic/configs/stateful/oblt.stateful.config.ts --grep="Synthetics Alerting" ```

…c#222371) Relocating tests added in elastic#221139 into separate feature flag configs, that are not run on MKI but only Kibana CI. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…222371) (#224588) # Backport This will backport the following commits from `main` to `8.19`: - [[FTR] add feature flags configs for deployment-agnostic tests (#222371)](#222371) Also cherry-picked a few required changes from #218725 which was not backported.  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  --------- Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>

github-actions bot added the author:obs-ux-management PRs authored by the obs ux management team label May 21, 2025

cesco-f force-pushed the migrate-synthetics-tests branch from 7af4b85 to 77d7f05 Compare May 22, 2025 10:08

cesco-f added release_note:skip Skip the PR/issue when compiling release notes backport:version Backport to applied version labels v8.19.0 labels May 22, 2025

cesco-f force-pushed the migrate-synthetics-tests branch 2 times, most recently from 77d7f05 to db0c5cb Compare May 22, 2025 11:14

cesco-f marked this pull request as ready for review May 22, 2025 11:14

cesco-f requested review from a team as code owners May 22, 2025 11:14

wayneseymour approved these changes May 22, 2025

View reviewed changes

cesco-f force-pushed the migrate-synthetics-tests branch 5 times, most recently from 47893c3 to e6a28bb Compare May 23, 2025 13:55

cesco-f force-pushed the migrate-synthetics-tests branch from e6a28bb to 9e23dd7 Compare May 26, 2025 06:17

cesco-f added the ci:build-serverless-image label May 26, 2025

cesco-f force-pushed the migrate-synthetics-tests branch 2 times, most recently from 24e27f4 to f98245c Compare May 26, 2025 08:08

cesco-f added ci:project-deploy-observability Create an Observability project and removed ci:build-serverless-image labels May 26, 2025

cesco-f force-pushed the migrate-synthetics-tests branch from f98245c to ad30646 Compare May 26, 2025 13:21

cesco-f force-pushed the migrate-synthetics-tests branch from ad30646 to dc8d196 Compare May 27, 2025 09:31

jcger approved these changes May 29, 2025

View reviewed changes

test(synthetics): migrated synthetics default rule tests to deploymen…

8970eb5

…t agnostic

mgiota reviewed May 29, 2025

View reviewed changes

mgiota approved these changes May 29, 2025

View reviewed changes

mgiota reviewed May 29, 2025

View reviewed changes

cesco-f merged commit fdebd18 into elastic:main May 29, 2025
10 checks passed

kibanamachine added the v9.1.0 label May 29, 2025

cesco-f deleted the migrate-synthetics-tests branch May 29, 2025 10:20

kibanamachine mentioned this pull request May 29, 2025

[TSVB/SCSS] Migrate _annotations_editor.scss file #220900

Merged

1 task

cesco-f mentioned this pull request May 29, 2025

[8.19] [Synthetics] Migrate synthetics alerting API tests to deployment agnostic solution (#221139) #221908

Merged

dmlemeshko mentioned this pull request Jun 3, 2025

[FTR] add feature flags configs for deployment-agnostic tests #222371

Merged

Conversation

cesco-f commented May 21, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Serverless

Stateful

Uh oh!

wayneseymour left a comment

Choose a reason for hiding this comment

Uh oh!

cesco-f commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented May 23, 2025

💚 Build Succeeded

Metrics [docs]

ESLint disabled line counts

Total ESLint disabled count

History

Uh oh!

github-actions bot commented May 26, 2025

🤖 GitHub comments

Uh oh!

elasticmachine commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Metrics [docs]

ESLint disabled line counts

Total ESLint disabled count

History

Uh oh!

mgiota May 29, 2025

Choose a reason for hiding this comment

Uh oh!

cesco-f May 29, 2025

Choose a reason for hiding this comment

Uh oh!

mgiota May 29, 2025

Choose a reason for hiding this comment

Uh oh!

mgiota left a comment

Choose a reason for hiding this comment

Uh oh!

mgiota commented May 29, 2025

Uh oh!

mgiota May 29, 2025

Choose a reason for hiding this comment

Uh oh!

cesco-f May 29, 2025

Choose a reason for hiding this comment

Uh oh!

mgiota May 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kibanamachine commented May 29, 2025

Uh oh!

kibanamachine commented May 29, 2025

💔 All backports failed

Manual backport

Questions ?

Uh oh!

cesco-f commented May 29, 2025

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

cesco-f commented May 21, 2025 •

edited by kibanamachine

Loading

cesco-f commented May 22, 2025 •

edited

Loading

elasticmachine commented May 29, 2025 •

edited

Loading