refactor a11y tests to group by data sets

[LeeDr]

Summary

Most of the a11y tests use sample flights data set. But most of them navigated to '/tutorial_directory/sampleData' and at least checked that the data was installed. And only one of them was removing the data. So I created an index file and moved the sample data loading to a before method there. 3 tests either use no data, or load different data and they're in a separate describe block.

Checklist

Delete any items that are not applicable to this PR.

• Unit or functional tests were updated or added to match the most common scenarios

[bhavyarm]

@elasticmachine merge upstream

[LeeDr]

@elasticmachine merge upstream

Unrelated Jest test failure:

console.log
usageCollection.reportUiCounter is unavailable. Ensure this is setup via .

  at x-pack/plugins/observability/public/hooks/use_track_metric.tsx:61:15

[LeeDr]

@elasticmachine merge upstream

[kibanamachine]

💛 Build succeeded, but was flaky

• continuous-integration/kibana-ci/pull-request

• Commit: bb696f7

• Storybooks Preview

• Flaky suites:

  • xpack-kibana-ciGroup10

---

Test Failures

Kibana Pipeline / general / Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/uptime/certificates·ts.Uptime app with generated data certificates with certs page "before each" hook for "displays certificates"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://github.com/elastic/kibana/issues/112900

[00:00:00]     │
[00:42:39]       └-: Uptime app
[00:42:39]         └-> "before all" hook in "Uptime app"
[00:42:39]         └-: with generated data
[00:42:39]           └-> "before all" hook in "with generated data"
[00:43:29]           └-: certificates
[00:43:29]             └-> "before all" hook in "certificates"
[00:43:43]             └-: with certs
[00:43:43]               └-> "before all" hook for "can navigate to cert page"
[00:43:43]               └-> "before all" hook for "can navigate to cert page"
[00:43:43]                 │ info [x-pack/test/functional/es_archives/uptime/blank] Loading "mappings.json"
[00:43:43]                 │ info [x-pack/test/functional/es_archives/uptime/blank] Loading "data.json"
[00:43:43]                 │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [heartbeat-8-generated-test] creating index, cause [api], templates [], shards [1]/[1]
[00:43:43]                 │ info [x-pack/test/functional/es_archives/uptime/blank] Created index "heartbeat-8-generated-test"
[00:43:43]                 │ debg [x-pack/test/functional/es_archives/uptime/blank] "heartbeat-8-generated-test" settings undefined
[00:43:43]                 │ debg TestSubjects.exists(uptimeSettingsToOverviewLink)
[00:43:43]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="uptimeSettingsToOverviewLink"]') with timeout=0
[00:43:43]                 │ debg --- retry.tryForTime error: [data-test-subj="uptimeSettingsToOverviewLink"] is not displayed
[00:43:44]                 │ debg navigating to uptime url: http://localhost:61181/app/uptime
[00:43:44]                 │ debg navigate to: http://localhost:61181/app/uptime
[00:43:44]                 │ debg browser[INFO] http://localhost:61181/app/uptime?_t=1633530259143 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:43:44]                 │
[00:43:44]                 │ debg browser[INFO] http://localhost:61181/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:43:44]                 │ debg ... sleep(700) start
[00:43:45]                 │ debg ... sleep(700) end
[00:43:45]                 │ debg returned from get, calling refresh
[00:43:45]                 │ debg browser[INFO] http://localhost:61181/app/uptime?_t=1633530259143 281 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:43:45]                 │
[00:43:45]                 │ debg browser[INFO] http://localhost:61181/bootstrap.js 41:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:43:46]                 │ debg currentUrl = http://localhost:61181/app/uptime
[00:43:46]                 │          appUrl = http://localhost:61181/app/uptime
[00:43:46]                 │ debg TestSubjects.find(kibanaChrome)
[00:43:46]                 │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:43:46]                 │ debg ... sleep(501) start
[00:43:46]                 │ERROR browser[SEVERE] http://localhost:61181/46844/bundles/plugin/uptime/8.0.0/uptime.chunk.1.js 2:3687 "API /api/uptime/monitor/list is not returning expected response, Invalid value \"undefined\" supplied to \"summaries.state.summaryPings.monitor.timespan.lt\" for response" Object
[00:43:46]                 │ debg ... sleep(501) end
[00:43:46]                 │ debg in navigateTo url = http://localhost:61181/app/uptime
[00:43:46]                 │ debg isGlobalLoadingIndicatorVisible
[00:43:46]                 │ debg TestSubjects.exists(globalLoadingIndicator)
[00:43:46]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:43:48]                 │ debg --- retry.tryForTime error: [data-test-subj="globalLoadingIndicator"] is not displayed
[00:43:48]                 │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:43:48]                 │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:43:48]                 │ debg TestSubjects.exists(uptimeOverviewPage)
[00:43:48]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="uptimeOverviewPage"]') with timeout=2000
[00:43:49]                 │ debg TestSubjects.click(superDatePickerApplyTimeButton)
[00:43:49]                 │ debg Find.clickByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:43:49]                 │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:43:49]               └-> can navigate to cert page
[00:43:49]                 └-> "before each" hook: global before each for "can navigate to cert page"
[00:43:49]                 └-> "before each" hook: delete settings for "can navigate to cert page"
[00:43:49]                   │ debg Deleting saved object {
[00:43:49]                   │        type: 'uptime-dynamic-settings',
[00:43:49]                   │        id: 'uptime-dynamic-settings-singleton'
[00:43:49]                   │      }/%s
[00:43:49]                 └-> "before each" hook: load heartbeat data for "can navigate to cert page"
[00:43:49]                   │ info [x-pack/test/functional/es_archives/uptime/blank] Loading "mappings.json"
[00:43:49]                   │ info [x-pack/test/functional/es_archives/uptime/blank] Loading "data.json"
[00:43:49]                   │ info [o.e.c.m.MetadataDeleteIndexService] [node-01] [heartbeat-8-generated-test/XdTdj8wfRpGjdNpnc5JgYw] deleting index
[00:43:49]                   │ info [x-pack/test/functional/es_archives/uptime/blank] Deleted existing index "heartbeat-8-generated-test"
[00:43:49]                   │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [heartbeat-8-generated-test] creating index, cause [api], templates [], shards [1]/[1]
[00:43:49]                   │ERROR browser[SEVERE] http://localhost:61181/46844/bundles/plugin/uptime/8.0.0/uptime.chunk.1.js 2:3687 "API /api/uptime/monitor/list is not returning expected response, Invalid value \"undefined\" supplied to \"summaries.state.summaryPings.monitor.timespan.lt\" for response" Object
[00:43:49]                   │ info [x-pack/test/functional/es_archives/uptime/blank] Created index "heartbeat-8-generated-test"
[00:43:49]                   │ debg [x-pack/test/functional/es_archives/uptime/blank] "heartbeat-8-generated-test" settings undefined
[00:43:49]                 └-> "before each" hook for "can navigate to cert page"
[00:43:49]                 │ debg isGlobalLoadingIndicatorVisible
[00:43:49]                 │ debg TestSubjects.exists(globalLoadingIndicator)
[00:43:49]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:43:51]                 │ debg --- retry.tryForTime error: [data-test-subj="globalLoadingIndicator"] is not displayed
[00:43:51]                 │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:43:51]                 │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:43:51]                 │ debg TestSubjects.exists(data-missing)
[00:43:51]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="data-missing"]') with timeout=2500
[00:43:54]                 │ debg --- retry.tryForTime error: [data-test-subj="data-missing"] is not displayed
[00:43:54]                 │ debg TestSubjects.missingOrFail(data-missing)
[00:43:54]                 │ debg Find.waitForDeletedByCssSelector('[data-test-subj="data-missing"]') with timeout=2500
[00:43:55]                 │ debg Find.existsByCssSelector('[href="/app/uptime/certificates"]') with timeout=2500
[00:43:55]                 │ debg TestSubjects.exists(uptimeCertificatesPage)
[00:43:55]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="uptimeCertificatesPage"]') with timeout=0
[00:43:55]                 │ debg --- retry.tryForTime error: [data-test-subj="uptimeCertificatesPage"] is not displayed
[00:43:55]                 │ debg Find.existsByCssSelector('[href="/app/uptime/certificates"]') with timeout=0
[00:43:55]                 │ debg Find.clickByCssSelector('[href="/app/uptime/certificates"]') with timeout=10000
[00:43:55]                 │ debg Find.findByCssSelector('[href="/app/uptime/certificates"]') with timeout=10000
[00:43:55]                 │ debg TestSubjects.exists(uptimeCertificatesPage)
[00:43:55]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="uptimeCertificatesPage"]') with timeout=120000
[00:43:55]                 └- ✓ pass  (6.3s) "Uptime app with generated data certificates with certs can navigate to cert page"
[00:43:55]               └-> "after each" hook: unload for "can navigate to cert page"
[00:43:55]                 │ info [x-pack/test/functional/es_archives/uptime/blank] Unloading indices from "mappings.json"
[00:43:55]                 │ info [o.e.c.m.MetadataDeleteIndexService] [node-01] [heartbeat-8-generated-test/UusVHHnGR9mHH0LLsJiaLA] deleting index
[00:43:55]                 │ info [x-pack/test/functional/es_archives/uptime/blank] Deleted existing index "heartbeat-8-generated-test"
[00:43:55]                 │ info [x-pack/test/functional/es_archives/uptime/blank] Unloading indices from "data.json"
[00:43:55]               └-: page
[00:43:55]                 └-> "before all" hook for "displays certificates"
[00:43:55]                 └-> displays certificates
[00:43:55]                   └-> "before each" hook: global before each for "displays certificates"
[00:43:55]                   └-> "before each" hook: delete settings for "displays certificates"
[00:43:55]                     │ debg Deleting saved object {
[00:43:55]                     │        type: 'uptime-dynamic-settings',
[00:43:55]                     │        id: 'uptime-dynamic-settings-singleton'
[00:43:55]                     │      }/%s
[00:43:56]                   └-> "before each" hook: load heartbeat data for "displays certificates"
[00:43:56]                     │ info [x-pack/test/functional/es_archives/uptime/blank] Loading "mappings.json"
[00:43:56]                     │ info [x-pack/test/functional/es_archives/uptime/blank] Loading "data.json"
[00:43:56]                     │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [heartbeat-8-generated-test] creating index, cause [api], templates [], shards [1]/[1]
[00:43:56]                     │ info [x-pack/test/functional/es_archives/uptime/blank] Created index "heartbeat-8-generated-test"
[00:43:56]                     │ debg [x-pack/test/functional/es_archives/uptime/blank] "heartbeat-8-generated-test" settings undefined
[00:43:56]                   └-> "before each" hook for "displays certificates"
[00:43:56]                   └-> "before each" hook for "displays certificates"
[00:43:56]                     │ debg TestSubjects.exists(uptimeCertificatesPage)
[00:43:56]                     │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="uptimeCertificatesPage"]') with timeout=0
[00:43:56]                     │ debg TestSubjects.click(superDatePickerApplyTimeButton)
[00:43:56]                     │ debg Find.clickByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:43:56]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:44:06]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:44:06]                     │      Wait timed out after 10035ms
[00:44:07]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:44:17]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:44:17]                     │      Wait timed out after 10008ms
[00:44:17]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:44:27]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:44:27]                     │      Wait timed out after 10052ms
[00:44:28]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:44:38]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:44:38]                     │      Wait timed out after 10032ms
[00:44:38]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:44:48]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:44:48]                     │      Wait timed out after 10029ms
[00:44:49]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:44:59]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:44:59]                     │      Wait timed out after 10054ms
[00:44:59]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:45:09]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:45:09]                     │      Wait timed out after 10048ms
[00:45:10]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:45:20]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:45:20]                     │      Wait timed out after 10033ms
[00:45:20]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:45:30]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:45:30]                     │      Wait timed out after 10049ms
[00:45:31]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:45:41]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:45:41]                     │      Wait timed out after 10057ms
[00:45:41]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:45:51]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:45:51]                     │      Wait timed out after 10036ms
[00:45:52]                     │ debg Find.findByCssSelector('[data-test-subj="superDatePickerApplyTimeButton"]') with timeout=10000
[00:46:02]                     │ debg --- retry.try error: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:46:02]                     │      Wait timed out after 10054ms
[00:46:03]                     │ info Taking screenshot "/dev/shm/workspace/parallel/18/kibana/x-pack/test/functional/screenshots/failure/Uptime app with generated data certificates with certs page _before each_ hook for _displays certificates_.png"
[00:46:03]                     │ info Current URL is: http://localhost:61181/app/uptime/certificates
[00:46:03]                     │ info Saving page source to: /dev/shm/workspace/parallel/18/kibana/x-pack/test/functional/failure_debug/html/Uptime app with generated data certificates with certs page _before each_ hook for _displays certificates_.html
[00:46:03]                     └- ✖ fail: Uptime app with generated data certificates with certs page "before each" hook for "displays certificates"
[00:46:03]                     │      Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
[00:46:03]                     │ Wait timed out after 10054ms
[00:46:03]                     │     at /dev/shm/workspace/parallel/18/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
[00:46:03]                     │     at runMicrotasks (<anonymous>)
[00:46:03]                     │     at processTicksAndRejections (internal/process/task_queues.js:95:5)
[00:46:03]                     │       at onFailure (/dev/shm/workspace/parallel/18/kibana/test/common/services/retry/retry_for_success.ts:17:9)
[00:46:03]                     │       at retryForSuccess (/dev/shm/workspace/parallel/18/kibana/test/common/services/retry/retry_for_success.ts:57:13)
[00:46:03]                     │       at RetryService.try (/dev/shm/workspace/parallel/18/kibana/test/common/services/retry/retry.ts:31:12)
[00:46:03]                     │       at Proxy.clickByCssSelector (/dev/shm/workspace/parallel/18/kibana/test/functional/services/common/find.ts:360:5)
[00:46:03]                     │       at TestSubjects.click (/dev/shm/workspace/parallel/18/kibana/test/functional/services/common/test_subjects.ts:105:5)
[00:46:03]                     │       at refreshApp (test/functional/services/uptime/navigation.ts:32:5)
[00:46:03]                     │       at Object.refreshApp (test/functional/services/uptime/navigation.ts:37:7)
[00:46:03]                     │       at Context.<anonymous> (test/functional/apps/uptime/certificates.ts:67:11)
[00:46:03]                     │       at Object.apply (/dev/shm/workspace/parallel/18/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:46:03]                     │ 
[00:46:03]                     │ 

Stack Trace

Error: retry.try timeout: TimeoutError: Waiting for element to be located By(css selector, [data-test-subj="superDatePickerApplyTimeButton"])
Wait timed out after 10054ms
    at /dev/shm/workspace/parallel/18/kibana/node_modules/selenium-webdriver/lib/webdriver.js:842:17
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at onFailure (/dev/shm/workspace/parallel/18/kibana/test/common/services/retry/retry_for_success.ts:17:9)
    at retryForSuccess (/dev/shm/workspace/parallel/18/kibana/test/common/services/retry/retry_for_success.ts:57:13)
    at RetryService.try (/dev/shm/workspace/parallel/18/kibana/test/common/services/retry/retry.ts:31:12)
    at Proxy.clickByCssSelector (/dev/shm/workspace/parallel/18/kibana/test/functional/services/common/find.ts:360:5)
    at TestSubjects.click (/dev/shm/workspace/parallel/18/kibana/test/functional/services/common/test_subjects.ts:105:5)
    at refreshApp (test/functional/services/uptime/navigation.ts:32:5)
    at Object.refreshApp (test/functional/services/uptime/navigation.ts:37:7)
    at Context.<anonymous> (test/functional/apps/uptime/certificates.ts:67:11)
    at Object.apply (/dev/shm/workspace/parallel/18/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)

---

Metrics [docs]

✅ unchanged

History

• 💔 Build #158127 failed 8420714
• 💔 Build #157977 failed 2daee5e
• 💔 Build #157828 failed 3beeb36

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[bhavyarm]

LGTM. Thanks @LeeDr for doing this!

[elasticmachine]

Pinging @elastic/kibana-accessibility (Project:Accessibility)

[kibanamachine]

💔 Backport failed

Status	Branch	Result
✅	7.x
❌	7.15	Commit could not be cherrypicked due to conflicts

Successful backport PRs will be merged automatically after passing CI.

To backport manually run:
node scripts/backport --pr 113876