Bring back most of the filtering tests, only bypass the known current issues by stacey-gammon · Pull Request #22787 · elastic/kibana

stacey-gammon · 2018-09-06T22:55:06Z

Bring back most of the filtering tests.

Note:
There is something really weird going on when the _dashboard_filtering test suite is run multiple times, another error is consistently hit in the _dashboard_snapshot suite.

Rather than tackle that weird issue here, I'm just going to work on getting the filtering tests back on so no more bugs sneak by, and can then investigate the other issue separately.

elasticmachine · 2018-09-07T00:19:01Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-09-07T02:40:17Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-07T13:18:38Z

unrelated failure:

Error: Expected to find "saveDashboardSuccess" toast after saving dashboard
    at DashboardPage.saveDashboard (test/functional/page_objects/dashboard_page.js:316:15)
    at <anonymous>
    at process._tickCallback (internal/process/next_tick.js:188:7)

~~I think I will have a fix for this one here: #22766~~

Hm, no, not sure what is going on with that failure. The screenshot shows that there is no toast but looks like the dashboard was saved (at least it's not in edit mode).... maybe it took so long to save that the toast disappeared?

jenkins, test this

elasticmachine · 2018-09-07T15:14:47Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-07T16:01:17Z

Same error: Error: Expected to find "saveDashboardSuccess" toast after saving dashboard. Having this PR be the culprit does not really make sense to me, based on the changes. 😕

jenkins, test this

elasticmachine · 2018-09-07T17:51:03Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-09-07T20:24:37Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-09T12:35:12Z

failed on UI Functional Tests.test/functional/apps/context/_filters·js.context app context filters should be toggleable via the filter bar

jenkins, test this

elasticmachine · 2018-09-09T14:23:30Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-09-10T16:01:00Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-10T21:38:18Z

another issue with the dashboard snapshot test not finding the save dashboard success toast.

jenkins, test this

elasticmachine · 2018-09-10T23:27:42Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-11T20:54:13Z

Failed on snapshot tests for a different reason, the comparison failed: /_dashboard_snapshots·js.dashboard app using current data dashboard snapshots compare area chart snapshot

elasticmachine · 2018-09-12T02:45:14Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-12T11:49:35Z

so weird. Well, lets just see if we can get these changes in, when run only once.

jenkins, test this

elasticmachine · 2018-09-12T13:27:42Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-12T13:40:19Z

retest

ppisljar · 2018-09-12T14:23:37Z

test/functional/apps/dashboard/_dashboard_filtering.js

why has this changed ?

Because I added a test to remove the saved filter above. I did this as clean up, so the test suite leaves the code just as it was when it started. I wanted to run the suite muiltiple times in a row and If I didn't do this, it'd fail the second time around because the filter is then actually saved on the instance (good or bad, I'm not reloading all the index data after each test suite to save time).

ppisljar

not sure why number of pie slices changed, apart from that LGTM, lets run the tests few more times to be sure its stable.

elasticmachine · 2018-09-12T14:55:13Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-12T15:18:21Z

Failed on UI Functional Tests.test/functional/apps/context/_filters·js.context app context filters should be toggleable via the filter bar

jenkins, test this

elasticmachine · 2018-09-12T16:57:19Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-12T16:59:32Z

retest

elasticmachine · 2018-09-12T18:34:47Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-12T19:01:25Z

retest

elasticmachine · 2018-09-12T20:34:14Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-13T00:45:51Z

failed on:

20:32:28    │ERROR [POST http://localhost:9515/session / {"desiredCapabilities":{},"requiredCapabilities":{}}] unknown error: Chrome failed to start: exited abnormally
20:32:28    │        (unknown error: DevToolsActivePort file doesn't exist)
20:32:28    │        (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
20:32:28    │        (Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 4.9.0-8-amd64 x86_64)
20:32:28    │          at Server._post (/var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/test/functional/services/remote/verbose_remote_logging.js:34:21)
20:32:28    │          at Server.createSession (/var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/node_modules/leadfoot/Server.js:362:15)
20:32:28    │          at attemptToCreateCommand (/var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/test/functional/services/remote/leadfoot_command.js:38:32)
20:32:28    │          at Promise.race (/var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/test/functional/services/remote/leadfoot_command.js:86:11)
20:32:28    │          at initLeadfootCommand (/var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/test/functional/services/remote/leadfoot_command.js:63:5)
20:32:28    │          at RemoteProvider (/var/lib/jenkins/workspace/elastic+kibana+pull-request+multijob-selenium/kibana/test/functional/services/remote/remote.js:40:29)
20:32:28    │          at <anonymous>
20:32:28    │ warn 1 processes left running, stop them with procs.stop(name): [ 'kibana' ]
20:32:29    │ info [kibana] exited with 0 after a minute
20:32:29 
20:32:29 Failure initializing 1 service(s)

will retest again after rebasing

… issues.

…a row.

elasticmachine · 2018-09-13T13:07:30Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-13T13:13:03Z

retest

elasticmachine · 2018-09-13T14:48:53Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-09-13T14:55:19Z

K, I'm merging. No filter failures, just other sporadic unrelated failures.

… issues (elastic#22787) * Bring back most of the filtering tests, only bypass the known current issues. * Clean up after the test suite run so it can be run multiple times in a row. * goal and guage issues were because of a bad cache

… issues (#22787) (#22996) * Bring back most of the filtering tests, only bypass the known current issues. * Clean up after the test suite run so it can be run multiple times in a row. * goal and guage issues were because of a bad cache

stacey-gammon added :Sharing labels Sep 6, 2018

stacey-gammon force-pushed the 2018-09-06-only-skip-known-issues branch from 9561362 to 74b18a3 Compare September 7, 2018 19:13

stacey-gammon force-pushed the 2018-09-06-only-skip-known-issues branch from 74b18a3 to 3964316 Compare September 10, 2018 14:10

stacey-gammon mentioned this pull request Sep 11, 2018

The future of our Functional Test Runner #22947

Closed

stacey-gammon force-pushed the 2018-09-06-only-skip-known-issues branch from 3964316 to 4b97728 Compare September 12, 2018 01:08

stacey-gammon requested a review from ppisljar September 12, 2018 13:40

ppisljar reviewed Sep 12, 2018

View reviewed changes

ppisljar approved these changes Sep 12, 2018

View reviewed changes

Stacey Gammon added 3 commits September 13, 2018 07:27

Bring back most of the filtering tests, only bypass the known current…

fe0b5ae

… issues.

Clean up after the test suite run so it can be run multiple times in …

b0152a0

…a row.

goal and guage issues were because of a bad cache

c5751e0

stacey-gammon force-pushed the 2018-09-06-only-skip-known-issues branch from 4b97728 to c5751e0 Compare September 13, 2018 11:27

stacey-gammon mentioned this pull request Sep 13, 2018

Fix a few dashboard flaky tests #22901

Closed

stacey-gammon merged commit 5e44e00 into elastic:master Sep 13, 2018

stacey-gammon mentioned this pull request Sep 13, 2018

[6.x] Bring back most of the filtering tests, only bypass the known current issues (#22787) #22996

Merged

liza-mae added failed-test A test failure on a tracked branch, potentially flaky-test and removed flaky-failing-test labels Nov 1, 2018

Conversation

stacey-gammon commented Sep 6, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Sep 7, 2018

💔 Build Failed

Uh oh!

elasticmachine commented Sep 7, 2018

💔 Build Failed

Uh oh!

stacey-gammon commented Sep 7, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Sep 7, 2018

💔 Build Failed

Uh oh!

stacey-gammon commented Sep 7, 2018

Uh oh!

elasticmachine commented Sep 7, 2018

💔 Build Failed

Uh oh!

elasticmachine commented Sep 7, 2018

💔 Build Failed

Uh oh!

stacey-gammon commented Sep 9, 2018

Uh oh!

elasticmachine commented Sep 9, 2018

💔 Build Failed

Uh oh!

elasticmachine commented Sep 10, 2018

💔 Build Failed

Uh oh!

stacey-gammon commented Sep 10, 2018

Uh oh!

elasticmachine commented Sep 10, 2018

💔 Build Failed

Uh oh!

stacey-gammon commented Sep 11, 2018

Uh oh!

elasticmachine commented Sep 12, 2018

💚 Build Succeeded

Uh oh!

stacey-gammon commented Sep 12, 2018

Uh oh!

elasticmachine commented Sep 12, 2018

💚 Build Succeeded

Uh oh!

stacey-gammon commented Sep 12, 2018

Uh oh!

ppisljar Sep 12, 2018

Choose a reason for hiding this comment

Uh oh!

stacey-gammon Sep 12, 2018

Choose a reason for hiding this comment

Uh oh!

ppisljar left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Sep 12, 2018

💔 Build Failed

Uh oh!

stacey-gammon commented Sep 12, 2018

Uh oh!

elasticmachine commented Sep 12, 2018

💚 Build Succeeded

Uh oh!

stacey-gammon commented Sep 12, 2018

Uh oh!

elasticmachine commented Sep 12, 2018

💚 Build Succeeded

Uh oh!

stacey-gammon commented Sep 12, 2018

Uh oh!

elasticmachine commented Sep 12, 2018

💔 Build Failed

Uh oh!

stacey-gammon commented Sep 13, 2018

Uh oh!

elasticmachine commented Sep 13, 2018

💚 Build Succeeded

stacey-gammon commented Sep 6, 2018 •

edited

Loading

stacey-gammon commented Sep 7, 2018 •

edited

Loading