[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage#226064
Merged
nreese merged 1 commit intoelastic:mainfrom Jul 2, 2025
Merged
[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage#226064nreese merged 1 commit intoelastic:mainfrom
nreese merged 1 commit intoelastic:mainfrom
Conversation
Contributor
Author
|
/ci |
Contributor
💚 Build Succeeded
Metrics [docs]
|
nreese
changed the title
nreese
changed the title
Contributor
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#8504[✅] src/platform/test/functional/apps/dashboard/group1/config.ts: 100/100 tests passed. |
nreese
added
Team:Presentation
Contributor
|
Pinging @elastic/kibana-presentation (Team:Presentation) |
Contributor
|
Starting backport for target branches: 8.19, 9.0, 9.1 https://github.com/elastic/kibana/actions/runs/16030869797 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 2, 2025
…ashboardLandingPage (elastic#226064) The current implementation of `onDashboardLandingPage` calls `this.listingTable.onListingPage('dashboard')`. `onListingPage` is inlined below for reference. The problem with this implementation is that when the test is not on the listing page, there is a 5000 milliseconds delay checking for the existence of a never-gonna-be-there element. ``` public async onListingPage(appName: AppName) { return await this.testSubjects.exists(`${appName}LandingPage`, { timeout: 5000, }); } ``` `onDashboardLandingPage` is used 20 times, while `onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which is used 137 times. A majority of these calls occur when the test is on `dashboards#/view` so the tests delay 5 seconds on these calls. Needless to say, our tests waste a lot of time determining if they are on the listing page. One case study is src/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts, which calls `gotoDashboardLandingPage` 11 times. Before this change, this test takes 4 minutes to run. With this change, this test takes 3 minutes to run which lines up with our expection that 11 x 5 seconds is about one minute. The PR resolves the issue by checking the URL instead of the DOM. This way, the test instantly knows if its on the landing page without any delays. Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504. Ran src/platform/test/functional/apps/dashboard/group1/config.ts, which includes dashboard_unsaved_listing.ts, to ensure this change is not flaky. I also ran the flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506 as a control to see how long it takes without these changes. The runs where 2 to 3 minutes faster with the changes. Control - no changes to `onDashboardLandingPage` <img width="408" alt="Screenshot 2025-07-01 at 4 45 04 PM" src="https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15" /> With changes to `onDashboardLandingPage` <img width="348" alt="Screenshot 2025-07-01 at 4 45 44 PM" src="https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681" /> (cherry picked from commit 0a761c9)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 2, 2025
…ashboardLandingPage (elastic#226064) The current implementation of `onDashboardLandingPage` calls `this.listingTable.onListingPage('dashboard')`. `onListingPage` is inlined below for reference. The problem with this implementation is that when the test is not on the listing page, there is a 5000 milliseconds delay checking for the existence of a never-gonna-be-there element. ``` public async onListingPage(appName: AppName) { return await this.testSubjects.exists(`${appName}LandingPage`, { timeout: 5000, }); } ``` `onDashboardLandingPage` is used 20 times, while `onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which is used 137 times. A majority of these calls occur when the test is on `dashboards#/view` so the tests delay 5 seconds on these calls. Needless to say, our tests waste a lot of time determining if they are on the listing page. One case study is src/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts, which calls `gotoDashboardLandingPage` 11 times. Before this change, this test takes 4 minutes to run. With this change, this test takes 3 minutes to run which lines up with our expection that 11 x 5 seconds is about one minute. The PR resolves the issue by checking the URL instead of the DOM. This way, the test instantly knows if its on the landing page without any delays. Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504. Ran src/platform/test/functional/apps/dashboard/group1/config.ts, which includes dashboard_unsaved_listing.ts, to ensure this change is not flaky. I also ran the flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506 as a control to see how long it takes without these changes. The runs where 2 to 3 minutes faster with the changes. Control - no changes to `onDashboardLandingPage` <img width="408" alt="Screenshot 2025-07-01 at 4 45 04 PM" src="https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15" /> With changes to `onDashboardLandingPage` <img width="348" alt="Screenshot 2025-07-01 at 4 45 44 PM" src="https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681" /> (cherry picked from commit 0a761c9)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 2, 2025
…ashboardLandingPage (elastic#226064) The current implementation of `onDashboardLandingPage` calls `this.listingTable.onListingPage('dashboard')`. `onListingPage` is inlined below for reference. The problem with this implementation is that when the test is not on the listing page, there is a 5000 milliseconds delay checking for the existence of a never-gonna-be-there element. ``` public async onListingPage(appName: AppName) { return await this.testSubjects.exists(`${appName}LandingPage`, { timeout: 5000, }); } ``` `onDashboardLandingPage` is used 20 times, while `onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which is used 137 times. A majority of these calls occur when the test is on `dashboards#/view` so the tests delay 5 seconds on these calls. Needless to say, our tests waste a lot of time determining if they are on the listing page. One case study is src/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts, which calls `gotoDashboardLandingPage` 11 times. Before this change, this test takes 4 minutes to run. With this change, this test takes 3 minutes to run which lines up with our expection that 11 x 5 seconds is about one minute. The PR resolves the issue by checking the URL instead of the DOM. This way, the test instantly knows if its on the landing page without any delays. Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504. Ran src/platform/test/functional/apps/dashboard/group1/config.ts, which includes dashboard_unsaved_listing.ts, to ensure this change is not flaky. I also ran the flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506 as a control to see how long it takes without these changes. The runs where 2 to 3 minutes faster with the changes. Control - no changes to `onDashboardLandingPage` <img width="408" alt="Screenshot 2025-07-01 at 4 45 04 PM" src="https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15" /> With changes to `onDashboardLandingPage` <img width="348" alt="Screenshot 2025-07-01 at 4 45 44 PM" src="https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681" /> (cherry picked from commit 0a761c9)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jul 2, 2025
…ng onDashboardLandingPage (#226064) (#226275) # Backport This will backport the following commits from `main` to `9.0`: - [[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)](#226064) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Nathan Reese","email":"reese.nathan@elastic.co"},"sourceCommit":{"committedDate":"2025-07-02T16:41:37Z","message":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)\n\nThe current implementation of `onDashboardLandingPage` calls\n`this.listingTable.onListingPage('dashboard')`. `onListingPage` is\ninlined below for reference. The problem with this implementation is\nthat when the test is not on the listing page, there is a 5000\nmilliseconds delay checking for the existence of a never-gonna-be-there\nelement.\n\n```\npublic async onListingPage(appName: AppName) {\n return await this.testSubjects.exists(`${appName}LandingPage`, {\n timeout: 5000,\n });\n }\n```\n\n`onDashboardLandingPage` is used 20 times, while\n`onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which\nis used 137 times. A majority of these calls occur when the test is on\n`dashboards#/view` so the tests delay 5 seconds on these calls. Needless\nto say, our tests waste a lot of time determining if they are on the\nlisting page.\n\nOne case study is\nsrc/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts,\nwhich calls `gotoDashboardLandingPage` 11 times. Before this change,\nthis test takes 4 minutes to run. With this change, this test takes 3\nminutes to run which lines up with our expection that 11 x 5 seconds is\nabout one minute.\n\nThe PR resolves the issue by checking the URL instead of the DOM. This\nway, the test instantly knows if its on the landing page without any\ndelays.\n\nFlaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504.\nRan src/platform/test/functional/apps/dashboard/group1/config.ts, which\nincludes dashboard_unsaved_listing.ts, to ensure this change is not\nflaky.\n\nI also ran the flaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506\nas a control to see how long it takes without these changes. The runs\nwhere 2 to 3 minutes faster with the changes.\n\nControl - no changes to `onDashboardLandingPage`\n<img width=\"408\" alt=\"Screenshot 2025-07-01 at 4 45 04 PM\"\nsrc=\"https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15\"\n/>\n\nWith changes to `onDashboardLandingPage`\n<img width=\"348\" alt=\"Screenshot 2025-07-01 at 4 45 44 PM\"\nsrc=\"https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681\"\n/>","sha":"0a761c9b5dd7a2aaa1afd251c8667946565244c1","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Presentation","release_note:skip","v9.0.0","backport:version","v9.1.0","v8.19.0","v9.2.0"],"title":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage","number":226064,"url":"https://github.com/elastic/kibana/pull/226064","mergeCommit":{"message":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)\n\nThe current implementation of `onDashboardLandingPage` calls\n`this.listingTable.onListingPage('dashboard')`. `onListingPage` is\ninlined below for reference. The problem with this implementation is\nthat when the test is not on the listing page, there is a 5000\nmilliseconds delay checking for the existence of a never-gonna-be-there\nelement.\n\n```\npublic async onListingPage(appName: AppName) {\n return await this.testSubjects.exists(`${appName}LandingPage`, {\n timeout: 5000,\n });\n }\n```\n\n`onDashboardLandingPage` is used 20 times, while\n`onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which\nis used 137 times. A majority of these calls occur when the test is on\n`dashboards#/view` so the tests delay 5 seconds on these calls. Needless\nto say, our tests waste a lot of time determining if they are on the\nlisting page.\n\nOne case study is\nsrc/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts,\nwhich calls `gotoDashboardLandingPage` 11 times. Before this change,\nthis test takes 4 minutes to run. With this change, this test takes 3\nminutes to run which lines up with our expection that 11 x 5 seconds is\nabout one minute.\n\nThe PR resolves the issue by checking the URL instead of the DOM. This\nway, the test instantly knows if its on the landing page without any\ndelays.\n\nFlaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504.\nRan src/platform/test/functional/apps/dashboard/group1/config.ts, which\nincludes dashboard_unsaved_listing.ts, to ensure this change is not\nflaky.\n\nI also ran the flaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506\nas a control to see how long it takes without these changes. The runs\nwhere 2 to 3 minutes faster with the changes.\n\nControl - no changes to `onDashboardLandingPage`\n<img width=\"408\" alt=\"Screenshot 2025-07-01 at 4 45 04 PM\"\nsrc=\"https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15\"\n/>\n\nWith changes to `onDashboardLandingPage`\n<img width=\"348\" alt=\"Screenshot 2025-07-01 at 4 45 44 PM\"\nsrc=\"https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681\"\n/>","sha":"0a761c9b5dd7a2aaa1afd251c8667946565244c1"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","9.1","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/226064","number":226064,"mergeCommit":{"message":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)\n\nThe current implementation of `onDashboardLandingPage` calls\n`this.listingTable.onListingPage('dashboard')`. `onListingPage` is\ninlined below for reference. The problem with this implementation is\nthat when the test is not on the listing page, there is a 5000\nmilliseconds delay checking for the existence of a never-gonna-be-there\nelement.\n\n```\npublic async onListingPage(appName: AppName) {\n return await this.testSubjects.exists(`${appName}LandingPage`, {\n timeout: 5000,\n });\n }\n```\n\n`onDashboardLandingPage` is used 20 times, while\n`onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which\nis used 137 times. A majority of these calls occur when the test is on\n`dashboards#/view` so the tests delay 5 seconds on these calls. Needless\nto say, our tests waste a lot of time determining if they are on the\nlisting page.\n\nOne case study is\nsrc/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts,\nwhich calls `gotoDashboardLandingPage` 11 times. Before this change,\nthis test takes 4 minutes to run. With this change, this test takes 3\nminutes to run which lines up with our expection that 11 x 5 seconds is\nabout one minute.\n\nThe PR resolves the issue by checking the URL instead of the DOM. This\nway, the test instantly knows if its on the landing page without any\ndelays.\n\nFlaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504.\nRan src/platform/test/functional/apps/dashboard/group1/config.ts, which\nincludes dashboard_unsaved_listing.ts, to ensure this change is not\nflaky.\n\nI also ran the flaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506\nas a control to see how long it takes without these changes. The runs\nwhere 2 to 3 minutes faster with the changes.\n\nControl - no changes to `onDashboardLandingPage`\n<img width=\"408\" alt=\"Screenshot 2025-07-01 at 4 45 04 PM\"\nsrc=\"https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15\"\n/>\n\nWith changes to `onDashboardLandingPage`\n<img width=\"348\" alt=\"Screenshot 2025-07-01 at 4 45 44 PM\"\nsrc=\"https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681\"\n/>","sha":"0a761c9b5dd7a2aaa1afd251c8667946565244c1"}}]}] BACKPORT--> Co-authored-by: Nathan Reese <reese.nathan@elastic.co>
Contributor
|
Starting backport for target branches: 8.19, 9.0, 9.1 https://github.com/elastic/kibana/actions/runs/16032979551 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 2, 2025
…ashboardLandingPage (elastic#226064) The current implementation of `onDashboardLandingPage` calls `this.listingTable.onListingPage('dashboard')`. `onListingPage` is inlined below for reference. The problem with this implementation is that when the test is not on the listing page, there is a 5000 milliseconds delay checking for the existence of a never-gonna-be-there element. ``` public async onListingPage(appName: AppName) { return await this.testSubjects.exists(`${appName}LandingPage`, { timeout: 5000, }); } ``` `onDashboardLandingPage` is used 20 times, while `onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which is used 137 times. A majority of these calls occur when the test is on `dashboards#/view` so the tests delay 5 seconds on these calls. Needless to say, our tests waste a lot of time determining if they are on the listing page. One case study is src/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts, which calls `gotoDashboardLandingPage` 11 times. Before this change, this test takes 4 minutes to run. With this change, this test takes 3 minutes to run which lines up with our expection that 11 x 5 seconds is about one minute. The PR resolves the issue by checking the URL instead of the DOM. This way, the test instantly knows if its on the landing page without any delays. Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504. Ran src/platform/test/functional/apps/dashboard/group1/config.ts, which includes dashboard_unsaved_listing.ts, to ensure this change is not flaky. I also ran the flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506 as a control to see how long it takes without these changes. The runs where 2 to 3 minutes faster with the changes. Control - no changes to `onDashboardLandingPage` <img width="408" alt="Screenshot 2025-07-01 at 4 45 04 PM" src="https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15" /> With changes to `onDashboardLandingPage` <img width="348" alt="Screenshot 2025-07-01 at 4 45 44 PM" src="https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681" /> (cherry picked from commit 0a761c9)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 2, 2025
…ashboardLandingPage (elastic#226064) The current implementation of `onDashboardLandingPage` calls `this.listingTable.onListingPage('dashboard')`. `onListingPage` is inlined below for reference. The problem with this implementation is that when the test is not on the listing page, there is a 5000 milliseconds delay checking for the existence of a never-gonna-be-there element. ``` public async onListingPage(appName: AppName) { return await this.testSubjects.exists(`${appName}LandingPage`, { timeout: 5000, }); } ``` `onDashboardLandingPage` is used 20 times, while `onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which is used 137 times. A majority of these calls occur when the test is on `dashboards#/view` so the tests delay 5 seconds on these calls. Needless to say, our tests waste a lot of time determining if they are on the listing page. One case study is src/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts, which calls `gotoDashboardLandingPage` 11 times. Before this change, this test takes 4 minutes to run. With this change, this test takes 3 minutes to run which lines up with our expection that 11 x 5 seconds is about one minute. The PR resolves the issue by checking the URL instead of the DOM. This way, the test instantly knows if its on the landing page without any delays. Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504. Ran src/platform/test/functional/apps/dashboard/group1/config.ts, which includes dashboard_unsaved_listing.ts, to ensure this change is not flaky. I also ran the flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506 as a control to see how long it takes without these changes. The runs where 2 to 3 minutes faster with the changes. Control - no changes to `onDashboardLandingPage` <img width="408" alt="Screenshot 2025-07-01 at 4 45 04 PM" src="https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15" /> With changes to `onDashboardLandingPage` <img width="348" alt="Screenshot 2025-07-01 at 4 45 44 PM" src="https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681" /> (cherry picked from commit 0a761c9)
Contributor
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jul 2, 2025
…ng onDashboardLandingPage (#226064) (#226276) # Backport This will backport the following commits from `main` to `9.1`: - [[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)](#226064) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Nathan Reese","email":"reese.nathan@elastic.co"},"sourceCommit":{"committedDate":"2025-07-02T16:41:37Z","message":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)\n\nThe current implementation of `onDashboardLandingPage` calls\n`this.listingTable.onListingPage('dashboard')`. `onListingPage` is\ninlined below for reference. The problem with this implementation is\nthat when the test is not on the listing page, there is a 5000\nmilliseconds delay checking for the existence of a never-gonna-be-there\nelement.\n\n```\npublic async onListingPage(appName: AppName) {\n return await this.testSubjects.exists(`${appName}LandingPage`, {\n timeout: 5000,\n });\n }\n```\n\n`onDashboardLandingPage` is used 20 times, while\n`onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which\nis used 137 times. A majority of these calls occur when the test is on\n`dashboards#/view` so the tests delay 5 seconds on these calls. Needless\nto say, our tests waste a lot of time determining if they are on the\nlisting page.\n\nOne case study is\nsrc/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts,\nwhich calls `gotoDashboardLandingPage` 11 times. Before this change,\nthis test takes 4 minutes to run. With this change, this test takes 3\nminutes to run which lines up with our expection that 11 x 5 seconds is\nabout one minute.\n\nThe PR resolves the issue by checking the URL instead of the DOM. This\nway, the test instantly knows if its on the landing page without any\ndelays.\n\nFlaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504.\nRan src/platform/test/functional/apps/dashboard/group1/config.ts, which\nincludes dashboard_unsaved_listing.ts, to ensure this change is not\nflaky.\n\nI also ran the flaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506\nas a control to see how long it takes without these changes. The runs\nwhere 2 to 3 minutes faster with the changes.\n\nControl - no changes to `onDashboardLandingPage`\n<img width=\"408\" alt=\"Screenshot 2025-07-01 at 4 45 04 PM\"\nsrc=\"https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15\"\n/>\n\nWith changes to `onDashboardLandingPage`\n<img width=\"348\" alt=\"Screenshot 2025-07-01 at 4 45 44 PM\"\nsrc=\"https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681\"\n/>","sha":"0a761c9b5dd7a2aaa1afd251c8667946565244c1","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Presentation","release_note:skip","v9.0.0","backport:version","v9.1.0","v8.19.0","v9.2.0"],"title":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage","number":226064,"url":"https://github.com/elastic/kibana/pull/226064","mergeCommit":{"message":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)\n\nThe current implementation of `onDashboardLandingPage` calls\n`this.listingTable.onListingPage('dashboard')`. `onListingPage` is\ninlined below for reference. The problem with this implementation is\nthat when the test is not on the listing page, there is a 5000\nmilliseconds delay checking for the existence of a never-gonna-be-there\nelement.\n\n```\npublic async onListingPage(appName: AppName) {\n return await this.testSubjects.exists(`${appName}LandingPage`, {\n timeout: 5000,\n });\n }\n```\n\n`onDashboardLandingPage` is used 20 times, while\n`onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which\nis used 137 times. A majority of these calls occur when the test is on\n`dashboards#/view` so the tests delay 5 seconds on these calls. Needless\nto say, our tests waste a lot of time determining if they are on the\nlisting page.\n\nOne case study is\nsrc/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts,\nwhich calls `gotoDashboardLandingPage` 11 times. Before this change,\nthis test takes 4 minutes to run. With this change, this test takes 3\nminutes to run which lines up with our expection that 11 x 5 seconds is\nabout one minute.\n\nThe PR resolves the issue by checking the URL instead of the DOM. This\nway, the test instantly knows if its on the landing page without any\ndelays.\n\nFlaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504.\nRan src/platform/test/functional/apps/dashboard/group1/config.ts, which\nincludes dashboard_unsaved_listing.ts, to ensure this change is not\nflaky.\n\nI also ran the flaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506\nas a control to see how long it takes without these changes. The runs\nwhere 2 to 3 minutes faster with the changes.\n\nControl - no changes to `onDashboardLandingPage`\n<img width=\"408\" alt=\"Screenshot 2025-07-01 at 4 45 04 PM\"\nsrc=\"https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15\"\n/>\n\nWith changes to `onDashboardLandingPage`\n<img width=\"348\" alt=\"Screenshot 2025-07-01 at 4 45 44 PM\"\nsrc=\"https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681\"\n/>","sha":"0a761c9b5dd7a2aaa1afd251c8667946565244c1"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","9.1","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/226064","number":226064,"mergeCommit":{"message":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)\n\nThe current implementation of `onDashboardLandingPage` calls\n`this.listingTable.onListingPage('dashboard')`. `onListingPage` is\ninlined below for reference. The problem with this implementation is\nthat when the test is not on the listing page, there is a 5000\nmilliseconds delay checking for the existence of a never-gonna-be-there\nelement.\n\n```\npublic async onListingPage(appName: AppName) {\n return await this.testSubjects.exists(`${appName}LandingPage`, {\n timeout: 5000,\n });\n }\n```\n\n`onDashboardLandingPage` is used 20 times, while\n`onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which\nis used 137 times. A majority of these calls occur when the test is on\n`dashboards#/view` so the tests delay 5 seconds on these calls. Needless\nto say, our tests waste a lot of time determining if they are on the\nlisting page.\n\nOne case study is\nsrc/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts,\nwhich calls `gotoDashboardLandingPage` 11 times. Before this change,\nthis test takes 4 minutes to run. With this change, this test takes 3\nminutes to run which lines up with our expection that 11 x 5 seconds is\nabout one minute.\n\nThe PR resolves the issue by checking the URL instead of the DOM. This\nway, the test instantly knows if its on the landing page without any\ndelays.\n\nFlaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504.\nRan src/platform/test/functional/apps/dashboard/group1/config.ts, which\nincludes dashboard_unsaved_listing.ts, to ensure this change is not\nflaky.\n\nI also ran the flaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506\nas a control to see how long it takes without these changes. The runs\nwhere 2 to 3 minutes faster with the changes.\n\nControl - no changes to `onDashboardLandingPage`\n<img width=\"408\" alt=\"Screenshot 2025-07-01 at 4 45 04 PM\"\nsrc=\"https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15\"\n/>\n\nWith changes to `onDashboardLandingPage`\n<img width=\"348\" alt=\"Screenshot 2025-07-01 at 4 45 44 PM\"\nsrc=\"https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681\"\n/>","sha":"0a761c9b5dd7a2aaa1afd251c8667946565244c1"}}]}] BACKPORT--> Co-authored-by: Nathan Reese <reese.nathan@elastic.co>
kibanamachine
added a commit
that referenced
this pull request
Jul 2, 2025
…ing onDashboardLandingPage (#226064) (#226274) # Backport This will backport the following commits from `main` to `8.19`: - [[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)](#226064) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Nathan Reese","email":"reese.nathan@elastic.co"},"sourceCommit":{"committedDate":"2025-07-02T16:41:37Z","message":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)\n\nThe current implementation of `onDashboardLandingPage` calls\n`this.listingTable.onListingPage('dashboard')`. `onListingPage` is\ninlined below for reference. The problem with this implementation is\nthat when the test is not on the listing page, there is a 5000\nmilliseconds delay checking for the existence of a never-gonna-be-there\nelement.\n\n```\npublic async onListingPage(appName: AppName) {\n return await this.testSubjects.exists(`${appName}LandingPage`, {\n timeout: 5000,\n });\n }\n```\n\n`onDashboardLandingPage` is used 20 times, while\n`onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which\nis used 137 times. A majority of these calls occur when the test is on\n`dashboards#/view` so the tests delay 5 seconds on these calls. Needless\nto say, our tests waste a lot of time determining if they are on the\nlisting page.\n\nOne case study is\nsrc/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts,\nwhich calls `gotoDashboardLandingPage` 11 times. Before this change,\nthis test takes 4 minutes to run. With this change, this test takes 3\nminutes to run which lines up with our expection that 11 x 5 seconds is\nabout one minute.\n\nThe PR resolves the issue by checking the URL instead of the DOM. This\nway, the test instantly knows if its on the landing page without any\ndelays.\n\nFlaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504.\nRan src/platform/test/functional/apps/dashboard/group1/config.ts, which\nincludes dashboard_unsaved_listing.ts, to ensure this change is not\nflaky.\n\nI also ran the flaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506\nas a control to see how long it takes without these changes. The runs\nwhere 2 to 3 minutes faster with the changes.\n\nControl - no changes to `onDashboardLandingPage`\n<img width=\"408\" alt=\"Screenshot 2025-07-01 at 4 45 04 PM\"\nsrc=\"https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15\"\n/>\n\nWith changes to `onDashboardLandingPage`\n<img width=\"348\" alt=\"Screenshot 2025-07-01 at 4 45 44 PM\"\nsrc=\"https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681\"\n/>","sha":"0a761c9b5dd7a2aaa1afd251c8667946565244c1","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Presentation","release_note:skip","v9.0.0","backport:version","v9.1.0","v8.19.0","v9.2.0"],"title":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage","number":226064,"url":"https://github.com/elastic/kibana/pull/226064","mergeCommit":{"message":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)\n\nThe current implementation of `onDashboardLandingPage` calls\n`this.listingTable.onListingPage('dashboard')`. `onListingPage` is\ninlined below for reference. The problem with this implementation is\nthat when the test is not on the listing page, there is a 5000\nmilliseconds delay checking for the existence of a never-gonna-be-there\nelement.\n\n```\npublic async onListingPage(appName: AppName) {\n return await this.testSubjects.exists(`${appName}LandingPage`, {\n timeout: 5000,\n });\n }\n```\n\n`onDashboardLandingPage` is used 20 times, while\n`onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which\nis used 137 times. A majority of these calls occur when the test is on\n`dashboards#/view` so the tests delay 5 seconds on these calls. Needless\nto say, our tests waste a lot of time determining if they are on the\nlisting page.\n\nOne case study is\nsrc/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts,\nwhich calls `gotoDashboardLandingPage` 11 times. Before this change,\nthis test takes 4 minutes to run. With this change, this test takes 3\nminutes to run which lines up with our expection that 11 x 5 seconds is\nabout one minute.\n\nThe PR resolves the issue by checking the URL instead of the DOM. This\nway, the test instantly knows if its on the landing page without any\ndelays.\n\nFlaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504.\nRan src/platform/test/functional/apps/dashboard/group1/config.ts, which\nincludes dashboard_unsaved_listing.ts, to ensure this change is not\nflaky.\n\nI also ran the flaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506\nas a control to see how long it takes without these changes. The runs\nwhere 2 to 3 minutes faster with the changes.\n\nControl - no changes to `onDashboardLandingPage`\n<img width=\"408\" alt=\"Screenshot 2025-07-01 at 4 45 04 PM\"\nsrc=\"https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15\"\n/>\n\nWith changes to `onDashboardLandingPage`\n<img width=\"348\" alt=\"Screenshot 2025-07-01 at 4 45 44 PM\"\nsrc=\"https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681\"\n/>","sha":"0a761c9b5dd7a2aaa1afd251c8667946565244c1"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","9.1","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/226064","number":226064,"mergeCommit":{"message":"[dashboard] avoid 5 second delay in functional tests when calling onDashboardLandingPage (#226064)\n\nThe current implementation of `onDashboardLandingPage` calls\n`this.listingTable.onListingPage('dashboard')`. `onListingPage` is\ninlined below for reference. The problem with this implementation is\nthat when the test is not on the listing page, there is a 5000\nmilliseconds delay checking for the existence of a never-gonna-be-there\nelement.\n\n```\npublic async onListingPage(appName: AppName) {\n return await this.testSubjects.exists(`${appName}LandingPage`, {\n timeout: 5000,\n });\n }\n```\n\n`onDashboardLandingPage` is used 20 times, while\n`onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which\nis used 137 times. A majority of these calls occur when the test is on\n`dashboards#/view` so the tests delay 5 seconds on these calls. Needless\nto say, our tests waste a lot of time determining if they are on the\nlisting page.\n\nOne case study is\nsrc/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts,\nwhich calls `gotoDashboardLandingPage` 11 times. Before this change,\nthis test takes 4 minutes to run. With this change, this test takes 3\nminutes to run which lines up with our expection that 11 x 5 seconds is\nabout one minute.\n\nThe PR resolves the issue by checking the URL instead of the DOM. This\nway, the test instantly knows if its on the landing page without any\ndelays.\n\nFlaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504.\nRan src/platform/test/functional/apps/dashboard/group1/config.ts, which\nincludes dashboard_unsaved_listing.ts, to ensure this change is not\nflaky.\n\nI also ran the flaky test runner\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506\nas a control to see how long it takes without these changes. The runs\nwhere 2 to 3 minutes faster with the changes.\n\nControl - no changes to `onDashboardLandingPage`\n<img width=\"408\" alt=\"Screenshot 2025-07-01 at 4 45 04 PM\"\nsrc=\"https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15\"\n/>\n\nWith changes to `onDashboardLandingPage`\n<img width=\"348\" alt=\"Screenshot 2025-07-01 at 4 45 44 PM\"\nsrc=\"https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681\"\n/>","sha":"0a761c9b5dd7a2aaa1afd251c8667946565244c1"}}]}] BACKPORT--> Co-authored-by: Nathan Reese <reese.nathan@elastic.co>
kertal
pushed a commit
to kertal/kibana
that referenced
this pull request
Jul 25, 2025
…ashboardLandingPage (elastic#226064) The current implementation of `onDashboardLandingPage` calls `this.listingTable.onListingPage('dashboard')`. `onListingPage` is inlined below for reference. The problem with this implementation is that when the test is not on the listing page, there is a 5000 milliseconds delay checking for the existence of a never-gonna-be-there element. ``` public async onListingPage(appName: AppName) { return await this.testSubjects.exists(`${appName}LandingPage`, { timeout: 5000, }); } ``` `onDashboardLandingPage` is used 20 times, while `onDashboardLandingPage` is called in `gotoDashboardLandingPage`, which is used 137 times. A majority of these calls occur when the test is on `dashboards#/view` so the tests delay 5 seconds on these calls. Needless to say, our tests waste a lot of time determining if they are on the listing page. One case study is src/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts, which calls `gotoDashboardLandingPage` 11 times. Before this change, this test takes 4 minutes to run. With this change, this test takes 3 minutes to run which lines up with our expection that 11 x 5 seconds is about one minute. The PR resolves the issue by checking the URL instead of the DOM. This way, the test instantly knows if its on the landing page without any delays. Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504. Ran src/platform/test/functional/apps/dashboard/group1/config.ts, which includes dashboard_unsaved_listing.ts, to ensure this change is not flaky. I also ran the flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506 as a control to see how long it takes without these changes. The runs where 2 to 3 minutes faster with the changes. Control - no changes to `onDashboardLandingPage` <img width="408" alt="Screenshot 2025-07-01 at 4 45 04 PM" src="https://github.com/user-attachments/assets/58acc42c-241c-4005-b564-c114ff6b7b15" /> With changes to `onDashboardLandingPage` <img width="348" alt="Screenshot 2025-07-01 at 4 45 44 PM" src="https://github.com/user-attachments/assets/45ed8cde-e78c-4be6-a6b4-e9d95626a681" />
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The current implementation of
onDashboardLandingPagecallsthis.listingTable.onListingPage('dashboard').onListingPageis inlined below for reference. The problem with this implementation is that when the test is not on the listing page, there is a 5000 milliseconds delay checking for the existence of a never-gonna-be-there element.onDashboardLandingPageis used 20 times, whileonDashboardLandingPageis called ingotoDashboardLandingPage, which is used 137 times. A majority of these calls occur when the test is ondashboards#/viewso the tests delay 5 seconds on these calls. Needless to say, our tests waste a lot of time determining if they are on the listing page.One case study is src/platform/test/functional/apps/dashboard/group1/dashboard_unsaved_listing.ts, which calls
gotoDashboardLandingPage11 times. Before this change, this test takes 4 minutes to run. With this change, this test takes 3 minutes to run which lines up with our expection that 11 x 5 seconds is about one minute.The PR resolves the issue by checking the URL instead of the DOM. This way, the test instantly knows if its on the landing page without any delays.
Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8504. Ran src/platform/test/functional/apps/dashboard/group1/config.ts, which includes dashboard_unsaved_listing.ts, to ensure this change is not flaky.
I also ran the flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8506 as a control to see how long it takes without these changes. The runs where 2 to 3 minutes faster with the changes.
Control - no changes to
onDashboardLandingPageWith changes to
onDashboardLandingPage