Move test utils from legacy api to agnostic by yuliia-fryshko · Pull Request #224381 · elastic/kibana

yuliia-fryshko · 2025-06-18T08:59:07Z

Summary

Closes 219684

This PR removes the observability_ai_assistant_api_integration folder, relocating its tests and dependencies into relevant functional test directories or agnostic api tests. Common utilities used across both (functional and agnostic) suites have been moved to the agnostic folder.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

See some risk examples
...

elasticmachine · 2025-06-18T08:59:13Z

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

github-actions · 2025-06-18T08:59:18Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

github-actions · 2025-06-18T09:23:07Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

jbudz

.buildkite/ftr_oblt_stateful_configs.yml LGTM

viduni94 · 2025-06-19T15:35:28Z

x-pack/test/observability_ai_assistant_functional/common/config.ts

+      getScopedApiClientForUsername: () => getScopedApiClientForUsername,
+      apmSynthtraceEsClient: (context: InheritedFtrProviderContext) =>
+        getApmSynthtraceEsClient(context, apmSynthtraceKibanaClient),
+      observabilityAIAssistantAPIClient: async () => {


Is this being used anywhere? If not, can we remove it?

viduni94 · 2025-06-19T15:43:22Z

...ment_agnostic/apis/observability/ai_assistant/utils/observability_ai_assistant_api_client.ts

Is this file needed? Are we using the API client in this anywhere?
The deployment agnostic API client is in x-pack/test/api_integration/deployment_agnostic/services/observability_ai_assistant_api.ts

viduni94 · 2025-06-19T15:46:11Z

.github/CODEOWNERS

 /x-pack/test_serverless/functional/page_objects/svl_data_usage.ts @elastic/kibana-management
-/x-pack/test/observability_ai_assistant_api_integration @elastic/obs-ai-assistant
 /x-pack/test/observability_ai_assistant_functional @elastic/obs-ai-assistant
 /x-pack/test_serverless/**/test_suites/observability/ai_assistant @elastic/obs-ai-assistant


Can we remove everything in test_serverless too since we moved to deployment_agnostic?

csr · 2025-06-23T11:06:35Z

x-pack/test/observability_ai_assistant_functional/common/config.ts

+        getApmSynthtraceEsClient(context, apmSynthtraceKibanaClient),
+    },
+    junit: {
+      reportName: `Observability AI Assistant API Integration tests (${name})`,


It looks like we're creating an API integration test config here in the functional test plugin. AFAIK, unlike UI tests, API integration tests aren't considered functional tests. Is this intended?

csr · 2025-06-23T11:13:14Z

x-pack/test/observability_ai_assistant_functional/common/config.ts

@@ -5,29 +5,108 @@
 * 2.0.


In line with the Sustainable Kibana Architecture we're aiming to move solution-specific x-pack/test plugins to the x-pack/solutions/SOLUTION/test directory (this is the related observability directory). Could we please move this plugin to that directory?

We talked offline and agreed to leave this item as a follow-up. Thank you!

viduni94

Thanks for removing the test_serverless/**/test_suites/observability/ai_assistant test folder @yuliia-fryshko

Can you please remove all references to it as well?
I see it being referred in

CODEOWNERS file
.buildkite/ftr_oblt_serverless_configs.yml file

viduni94 · 2025-06-23T15:44:52Z

x-pack/test/observability_ai_assistant_functional/common/config.ts

 export type CreateTestConfig = ReturnType<typeof createTestConfig>;
 export type TestConfig = Awaited<ReturnType<typeof getTestConfig>>;

+export function createObservabilityAIAssistantAPIConfig({


Why do we have a createObservabilityAIAssistantAPIConfig and getTestConfig?
AFAICT, only apmSynthtraceEsClient from createObservabilityAIAssistantAPIConfig is being used in functional tests (correct me if I'm wrong). If that's the case, can we move apmSynthtraceEsClient to getTestConfig and move everything else that's needed from the base config to getTestConfig and completely get rid of createObservabilityAIAssistantAPIConfig? Would that work?

sorenlouv

thank you!

elasticmachine · 2025-06-27T18:46:33Z

⏳ Build in-progress

Buildkite Build
Commit: f91c3c7
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-224381-f91c3c75a449

History

💚 Build #313351 succeeded 2a1ec96
💛 Build #313301 was flaky b6af243
💚 Build #313025 succeeded 3a4b88a
💚 Build #312977 succeeded a65cd9f
💛 Build #312919 was flaky 7201279
💔 Build #312873 failed a4aa783

cc @yuliia-fryshko

kibanamachine · 2025-06-30T19:48:52Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 224381 locally
cc: @yuliia-fryshko

kibanamachine · 2025-07-01T20:49:12Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 224381 locally
cc: @yuliia-fryshko

kibanamachine · 2025-07-02T21:48:36Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 224381 locally
cc: @yuliia-fryshko

kibanamachine · 2025-07-03T21:48:55Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 224381 locally
cc: @yuliia-fryshko

kibanamachine · 2025-07-04T21:26:47Z

Starting backport for target branches: 9.1

https://github.com/elastic/kibana/actions/runs/16081123349

## Summary Closes [219684](elastic#219684) This PR removes the `observability_ai_assistant_api_integration` folder, relocating its tests and dependencies into relevant functional test directories or agnostic api tests. Common utilities used across both (functional and agnostic) suites have been moved to the agnostic folder. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 994ebc0)

kibanamachine · 2025-07-04T21:33:00Z

💚 All backports created successfully

Status	Branch	Result
✅	9.1

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine · 2025-07-04T22:38:35Z

Starting backport for target branches: 8.19, 9.1

https://github.com/elastic/kibana/actions/runs/16081790737

## Summary Closes [219684](elastic#219684) This PR removes the `observability_ai_assistant_api_integration` folder, relocating its tests and dependencies into relevant functional test directories or agnostic api tests. Common utilities used across both (functional and agnostic) suites have been moved to the agnostic folder. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 994ebc0)

kibanamachine · 2025-07-04T22:45:11Z

💔 Some backports could not be created

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts
✅	9.1

Note: Successful backport PRs will be merged automatically after passing CI.

Manual backport

To create the backport manually run:

node scripts/backport --pr 224381

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `9.1`: - [Move test utils from legacy api to agnostic (#224381)](#224381)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)  Co-authored-by: Yuliia Fryshko <yuliia.fryshko@elastic.co>

Closes [219684](elastic#219684) This PR removes the `observability_ai_assistant_api_integration` folder, relocating its tests and dependencies into relevant functional test directories or agnostic api tests. Common utilities used across both (functional and agnostic) suites have been moved to the agnostic folder. Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 994ebc0)

Backport This will backport the following commits from main to 8.19: #224381 Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

yuliia-fryshko requested review from a team as code owners June 18, 2025 08:59

botelastic bot added ci:project-deploy-observability Create an Observability project Team:Obs AI Assistant Observability AI Assistant labels Jun 18, 2025

yuliia-fryshko added backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes and removed ci:project-deploy-observability Create an Observability project labels Jun 18, 2025

botelastic bot added the ci:project-deploy-observability Create an Observability project label Jun 18, 2025

jbudz approved these changes Jun 18, 2025

View reviewed changes

yuliia-fryshko force-pushed the move-test-utils-from-legacy-api-to-agnostic branch from 60c2a7e to 885cb9b Compare June 18, 2025 15:54

TattdCodeMonkey approved these changes Jun 18, 2025

View reviewed changes

yuliia-fryshko force-pushed the move-test-utils-from-legacy-api-to-agnostic branch 2 times, most recently from 084845d to d3606a2 Compare June 19, 2025 07:19

yuliia-fryshko marked this pull request as draft June 19, 2025 08:13

yuliia-fryshko self-assigned this Jun 19, 2025

yuliia-fryshko marked this pull request as ready for review June 19, 2025 15:00

viduni94 reviewed Jun 19, 2025

View reviewed changes

yuliia-fryshko force-pushed the move-test-utils-from-legacy-api-to-agnostic branch from 7ce5728 to 703348e Compare June 23, 2025 08:40

yuliia-fryshko requested a review from a team as a code owner June 23, 2025 08:40

csr self-requested a review June 23, 2025 08:44

yuliia-fryshko force-pushed the move-test-utils-from-legacy-api-to-agnostic branch from 6cd8fbd to 0347510 Compare June 23, 2025 09:46

csr reviewed Jun 23, 2025

View reviewed changes

csr approved these changes Jun 23, 2025

View reviewed changes

viduni94 reviewed Jun 23, 2025

View reviewed changes

yuliia-fryshko added 2 commits June 26, 2025 23:43

Renamed the report of functional tests

b6af243

Merge branch 'main' into move-test-utils-from-legacy-api-to-agnostic

2a1ec96

sorenlouv approved these changes Jun 27, 2025

View reviewed changes

yuliia-fryshko added 2 commits June 27, 2025 18:53

Merge branch 'main' into move-test-utils-from-legacy-api-to-agnostic

de2a289

Merge branch 'main' into move-test-utils-from-legacy-api-to-agnostic

f91c3c7

yuliia-fryshko enabled auto-merge (squash) June 27, 2025 17:44

yuliia-fryshko merged commit 994ebc0 into elastic:main Jun 27, 2025
10 checks passed

kibanamachine added v9.2.0 backport missing Added to PRs automatically when the are determined to be missing a backport. labels Jun 27, 2025

sorenlouv added the v9.1.0 label Jul 4, 2025

kibanamachine mentioned this pull request Jul 4, 2025

[9.1] Move test utils from legacy api to agnostic (#224381) #226667

Merged

sorenlouv added the v8.19.0 label Jul 4, 2025

kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Jul 5, 2025

kibanamachine mentioned this pull request Jul 7, 2025

[Obs AI Assistant] Migrate anonymization to inference #225539

Merged

yuliia-fryshko mentioned this pull request Jul 8, 2025

Move test utils from legacy api to agnostic (#224381) #226967

Merged

kibanamachine mentioned this pull request Jul 8, 2025

[ska] relocate functional platform tests #226470

Merged

Comments

Conversation

yuliia-fryshko commented Jun 18, 2025 • edited by kibanamachine Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Identify risks

Uh oh!

elasticmachine commented Jun 18, 2025

Uh oh!

github-actions bot commented Jun 18, 2025

🤖 GitHub comments

Uh oh!

github-actions bot commented Jun 18, 2025

🤖 GitHub comments

Uh oh!

jbudz left a comment

Choose a reason for hiding this comment

Uh oh!

viduni94 Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

viduni94 Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

viduni94 Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

csr Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

csr Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

csr Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

viduni94 left a comment

Choose a reason for hiding this comment

Uh oh!

viduni94 Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

sorenlouv left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jun 27, 2025

⏳ Build in-progress

History

Uh oh!

Uh oh!

kibanamachine commented Jun 30, 2025

Uh oh!

kibanamachine commented Jul 1, 2025

Uh oh!

kibanamachine commented Jul 2, 2025

Uh oh!

kibanamachine commented Jul 3, 2025

Uh oh!

kibanamachine commented Jul 4, 2025

Uh oh!

kibanamachine commented Jul 4, 2025

💚 All backports created successfully

Questions ?

Uh oh!

kibanamachine commented Jul 4, 2025

Uh oh!

kibanamachine commented Jul 4, 2025

💔 Some backports could not be created

Manual backport

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

yuliia-fryshko commented Jun 18, 2025 •

edited by kibanamachine

Loading