[AI Infra] Fix failing GenAI Settings Scout tests#260496
Merged
KodeRad merged 16 commits intoApr 22, 2026
Merged
Conversation
Contributor
Author
|
/ci |
1 similar comment
Contributor
Author
|
/ci |
Contributor
Author
|
/ci |
KodeRad
added
ci:cloud-deploy
KodeRad
added
ci:cloud-deploy
Contributor
Author
|
/flaky scoutConfig:x-pack/platform/plugins/private/gen_ai_settings/test/scout/ui/parallel.playwright.config.ts:25 |
Contributor
Flaky Test Runner✅ Build triggered - kibana-flaky-test-suite-runner#11347
|
Contributor
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#11347[✅] x-pack/platform/plugins/private/gen_ai_settings/test/scout/ui/parallel.playwright.config.ts: 25/25 tests passed. |
KodeRad
changed the title
KodeRad
added
release_note:skip
Contributor
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
This was referenced Apr 22, 2026
mbondyra
added a commit
to mbondyra/kibana
that referenced
this pull request
Apr 22, 2026
…sationChanges23 * commit '9a7b717c662d1c904052bc59f0e5a81daab87c7f': (145 commits) Upgrade EUI to v114.2.0 (elastic#264550) [Entity Analytics] Add missing OpenAPI descriptions and examples to p… (elastic#264778) [Entity Resolution] Clarify CSV upload result for already-linked entities (elastic#264689) [AI Infra] Fix failing GenAI Settings Scout tests (elastic#260496) [Agent Builder] [Bug Bash] OAuth connector settings mention fields that are not there (elastic#264756) [performance] process-wide cache for advanced settings lookup (elastic#262618) [CI] Update limits.yml for securitySolution (elastic#264946) [SLO] Fix APM embeddable ids (elastic#264750) [EDR Workflows] Unify artifacts empty state buttons (elastic#264389) [Alert Triage workflow] Adds security.buildAlertEntityGraph and security.renderAlertNarrative… (elastic#259159) [SigEvents] Add KI feature identification endpoints and refactor task to use shared service (elastic#263528) [Scout] Migrate Data Views API tests from FTR - Part5 (elastic#264088) [Cases] Apply shared extended_fields path util server side (elastic#264706) [Lens as code] Fix metric trendline (elastic#264777) [api-docs] 2026-04-22 Daily api_docs build (elastic#264882) [Scout] Update test config manifests (elastic#264575) [workflows_management] Lazy-load Zod connector schemas to cut idle memory (elastic#264283) [ES|QL] Fix ES|QL columns reset race during active fetch (elastic#263947) [Content List] Column layout props, sticky actions, and title click handlers (elastic#264203) [Lens as code] Validate `id` in route for new vis types (elastic#264480) ...
Contributor
Author
This will not be backported to |
kibanamachine
added
the
backport missing
Contributor
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
1 similar comment
Contributor
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
4 tasks
SoniaSanzV
pushed a commit
to SoniaSanzV/kibana
that referenced
this pull request
Apr 27, 2026
## Summary This PR fixes the issues with failing GenAI Settings Scout test: elastic#260169 elastic#260019 elastic#259689 elastic#258770 elastic#261872 elastic#264278 GenAI Settings previously called management.sections.section.ai.registerApp() only after async work in setup() (getStartServices, licensing). That deferred registration could lose the race on slower VMs: Stack Management could build its route list before the app existed, so deep links to GenAI Settings sometimes landed on the management landing page while navigation still showed the item. Other management apps typically register synchronously in setup() and gate visibility separately. We now register the management app immediately in setup(), default it to disabled, and enable/disable it from start() based on enterprise license and connector/anonymization capabilities (with a license subscription for changes). There is a retry logic added to the scout test, so it can wait for the license to resolve and then retry. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: James Gowdy <jgowdy@elastic.co>
Contributor
|
Friendly reminder: Looks like this PR hasn’t been backported yet. |
KodeRad
removed
backport missing
KodeRad
added a commit
that referenced
this pull request
Apr 28, 2026
## Summary Extends the timeout values in the `GenAiSettingsPage.navigateTo()` page object to give the page sufficient time to load. The retry logic introduced in #260496 used a 3s inner `waitForSelector` timeout, which was too short on slower VMs — the page could fail to appear within that window even though it would have loaded given more time. This meant the retry loop was bailing out prematurely rather than waiting long enough per attempt. Changes: - `waitForSelector` timeout: `3_000` → `10_000` ms - `.toPass` outer timeout: `30_000` → `35_000` ms (accommodates 3 full retry attempts of up to 10s each, with intervals of 500ms / 1s / 2s between them) Relates to #260496 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [x] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Low risk — test-only change, no production code affected.
smith
pushed a commit
to smith/kibana
that referenced
this pull request
Apr 28, 2026
…ic#265692) ## Summary Extends the timeout values in the `GenAiSettingsPage.navigateTo()` page object to give the page sufficient time to load. The retry logic introduced in elastic#260496 used a 3s inner `waitForSelector` timeout, which was too short on slower VMs — the page could fail to appear within that window even though it would have loaded given more time. This meant the retry loop was bailing out prematurely rather than waiting long enough per attempt. Changes: - `waitForSelector` timeout: `3_000` → `10_000` ms - `.toPass` outer timeout: `30_000` → `35_000` ms (accommodates 3 full retry attempts of up to 10s each, with intervals of 500ms / 1s / 2s between them) Relates to elastic#260496 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [x] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks Low risk — test-only change, no production code affected.
This was referenced May 5, 2026
KodeRad
added a commit
that referenced
this pull request
May 6, 2026
…265692) (#267990) # Backport This will backport the following commits from `main` to `9.4`: - [[AI Infra] Extend GenAI Settings Scout page navigation timeout (#265692)](#265692) <!--- Backport version: 11.0.2 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Konrad Krasocki","email":"104936644+KodeRad@users.noreply.github.com"},"sourceCommit":{"committedDate":"2026-04-28T11:24:53Z","message":"[AI Infra] Extend GenAI Settings Scout page navigation timeout (#265692)\n\n## Summary\n\nExtends the timeout values in the `GenAiSettingsPage.navigateTo()` page\nobject to give the page sufficient time to load.\n\nThe retry logic introduced in #260496 used a 3s inner `waitForSelector`\ntimeout, which was too short on slower VMs — the page could fail to\nappear within that window even though it would have loaded given more\ntime. This meant the retry loop was bailing out prematurely rather than\nwaiting long enough per attempt.\n\nChanges:\n- `waitForSelector` timeout: `3_000` → `10_000` ms\n- `.toPass` outer timeout: `30_000` → `35_000` ms (accommodates 3 full\nretry attempts of up to 10s each, with intervals of 500ms / 1s / 2s\nbetween them)\n\nRelates to #260496\n\n### Checklist\n\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [x] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nLow risk — test-only change, no production code affected.","sha":"5462ceb8c6ccb0e582719b2dc432bcb09236b69f","branchLabelMapping":{"^v9.5.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":[":ml","release_note:skip","Team:ML","backport:version","reviewer:coderabbit","reviewer:macroscope","v9.5.0","v9.4.1"],"title":"[AI Infra] Extend GenAI Settings Scout page navigation timeout","number":265692,"url":"https://github.com/elastic/kibana/pull/265692","mergeCommit":{"message":"[AI Infra] Extend GenAI Settings Scout page navigation timeout (#265692)\n\n## Summary\n\nExtends the timeout values in the `GenAiSettingsPage.navigateTo()` page\nobject to give the page sufficient time to load.\n\nThe retry logic introduced in #260496 used a 3s inner `waitForSelector`\ntimeout, which was too short on slower VMs — the page could fail to\nappear within that window even though it would have loaded given more\ntime. This meant the retry loop was bailing out prematurely rather than\nwaiting long enough per attempt.\n\nChanges:\n- `waitForSelector` timeout: `3_000` → `10_000` ms\n- `.toPass` outer timeout: `30_000` → `35_000` ms (accommodates 3 full\nretry attempts of up to 10s each, with intervals of 500ms / 1s / 2s\nbetween them)\n\nRelates to #260496\n\n### Checklist\n\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [x] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nLow risk — test-only change, no production code affected.","sha":"5462ceb8c6ccb0e582719b2dc432bcb09236b69f"}},"sourceBranch":"main","suggestedTargetBranches":["9.4"],"targetPullRequestStates":[{"branch":"main","label":"v9.5.0","branchLabelMappingKey":"^v9.5.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/265692","number":265692,"mergeCommit":{"message":"[AI Infra] Extend GenAI Settings Scout page navigation timeout (#265692)\n\n## Summary\n\nExtends the timeout values in the `GenAiSettingsPage.navigateTo()` page\nobject to give the page sufficient time to load.\n\nThe retry logic introduced in #260496 used a 3s inner `waitForSelector`\ntimeout, which was too short on slower VMs — the page could fail to\nappear within that window even though it would have loaded given more\ntime. This meant the retry loop was bailing out prematurely rather than\nwaiting long enough per attempt.\n\nChanges:\n- `waitForSelector` timeout: `3_000` → `10_000` ms\n- `.toPass` outer timeout: `30_000` → `35_000` ms (accommodates 3 full\nretry attempts of up to 10s each, with intervals of 500ms / 1s / 2s\nbetween them)\n\nRelates to #260496\n\n### Checklist\n\n\n- [x] [Unit or functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere updated or added to match the most common scenarios\n- [x] [Flaky Test\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\nused on any tests changed\n- [x] The PR description includes the appropriate Release Notes section,\nand the correct `release_note:*` label is applied per the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\n- [x] Review the [backport\nguidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)\nand apply applicable `backport:*` labels.\n\n### Identify risks\n\nLow risk — test-only change, no production code affected.","sha":"5462ceb8c6ccb0e582719b2dc432bcb09236b69f"}},{"branch":"9.4","label":"v9.4.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes the issues with failing GenAI Settings Scout test:
#260169
#260019
#259689
#258770
#261872
#264278
GenAI Settings previously called management.sections.section.ai.registerApp() only after async work in setup() (getStartServices, licensing). That deferred registration could lose the race on slower VMs: Stack Management could build its route list before the app existed, so deep links to GenAI Settings sometimes landed on the management landing page while navigation still showed the item.
Other management apps typically register synchronously in setup() and gate visibility separately. We now register the management app immediately in setup(), default it to disabled, and enable/disable it from start() based on enterprise license and connector/anonymization capabilities (with a license subscription for changes).
There is a retry logic added to the scout test, so it can wait for the license to resolve and then retry.
Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
release_note:breakinglabel should be applied in these situations.release_note:*label is applied per the guidelinesbackport:*labels.Identify risks
Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.
Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.