Support Cloud Kibana UI testing by LeeDr · Pull Request #8880 · elastic/kibana

LeeDr · 2016-10-28T19:28:48Z

Support Cloud Kibana testing; Fixes #8861

tested setting environment variables for all elasticsearch and kibana protocol/host/port/user/pass settings.
Renamed "TEST_UI__" and "SHIELD__" variables to just be "TEST_*" to make life easier when setting these variables.
Browsers don't show ports 80 or 443 in the address bar, so filter them out of the expected URL in the navigateToApp function and in the link sharing tests.
Updated CONTRIBUTING.md to describe how to run tests against some other instance.
added a try loop around a failed data table test (not really related to Cloud testing but it failed for me once).
Added 'x-pack' "test" to check for the X-Pack Welcome banner and, if there, opt out, and dismiss. It's in a before method and so doesn't count as a test.
Instead of looking for the X-Pack welcome banner with some multi-second timeout, I got the outer toaster-container and got the text from it. Getting this and checking if the text contains 'X-Pack' happens in about 0.1s compared to a find timeout for something in the banner with a default of 10s.

I moved the run:devChromeDriver grunt task from the test:ui:server task to the test:ui:runner task since then you only need to run test:ui:runner against some already running Elasticsearch and Kibana servers.

If anyone wants to run this against my Cloud 5.0 instance I can give you the configuration settings.

LeeDr · 2016-10-28T19:30:51Z

This is in the CONTRIBUTING doc as an example for testing against Cloud;

export TEST_KIBANA_PROTOCOL=https
export TEST_KIBANA_HOSTNAME=9249d04b1186b3e7bbe11ea60df4f963.us-east-1.aws.found.io
export TEST_KIBANA_PORT=443
export TEST_KIBANA_USER=elastic
export TEST_KIBANA_PASS=<ping me for this>

export TEST_ES_PROTOCOL=http
export TEST_ES_HOSTNAME=aaa5d22032d76805fcce724ed9d9f5a2.us-east-1.aws.found.io
export TEST_ES_PORT=9200
export TEST_ES_USER=elastic
export TEST_ES_PASS=<ping me for this>
npm run test:ui:runner

rashmivkulkarni · 2016-10-29T00:35:43Z

To test this PR:

Checked out 5.0 Kibana : git clone https://github.com/elastic/kibana.git
Set the Upstream to 8880(PR)
git remote add upstream git@github.com:elastic/kibana.git
git Fetch the PR
git fetch upstream pull/8880/head && git checkout FETCH_HEAD
Do npm install
Do node --version : it should be v6.9.0 if old just do ( nvm install "$(cat .node-version)" to update the version)
Do nvm --version : 0.31.1
Update the run with the following cloud settings

export TEST_KIBANA_PROTOCOL=https
export TEST_KIBANA_HOSTNAME=f1b2798bb2ec1343528fb27091a0be72.us-east-1.aws.found.io 
export TEST_KIBANA_PORT=443
export TEST_KIBANA_USER=elastic
export TEST_KIBANA_PASS=kA1JiDX3czpjKXo7EOR4rQjt 
export TEST_ES_PROTOCOL=http
export TEST_ES_HOSTNAME=8fa6cf4361622605751460375efe1d17.us-east-1.aws.found.io
export TEST_ES_PORT=9200
export TEST_ES_USER=elastic
export TEST_ES_PASS=kA1JiDX3czpjKXo7EOR4rQjt

Do a npm run test:ui:runner and check out the results of the PR

jbudz · 2016-10-31T13:58:05Z

test/shield.js

 exports.kibanaServer = {
-  username: env.SHIELD_KIBANA_SERVER || 'kibana',
-  password: env.SHIELD_KIBANA_SERVER_PASS || 'notsecure'
+  username: env.TEST_KIBANA_SERVER_USER || 'elastic',


should this default to kibana?

Agreed, fixed.

jbudz · 2016-10-31T13:59:10Z

test/support/page_objects/common.js

+              if (kibanaApp === 'login') {
+                self.debug('Found loginPage username = '
+                  + config.servers.kibana.username
+                  + 'password = ' + config.servers.kibana.password);


can we remove password logging?

Agreed, I removed the password logging.

jbudz · 2016-10-31T13:59:58Z

test/support/page_objects/header_page.js


  isTimepickerOpen() {
-    return this.remote.setFindTimeout(defaultFindTimeout)
+    return this.remote.setFindTimeout(2000)


intentional?

Yes. When the timepicker is open we find it right away. When it's not open we don't want to wait the 10 second defaultFindTimeout.

jbudz · 2016-10-31T14:03:30Z

This merge is targeting 5.0, is the plan to forward port it to master?

jbudz · 2016-10-31T14:21:11Z

test/functional/apps/discover/_shared_links.js

@@ -20,6 +20,11 @@ bdd.describe('shared links', function describeIndexTests() {

  bdd.before(function () {
    baseUrl = PageObjects.common.getHostPort();


thoughts on replacing this with a common.getHost method?

Are you suggesting that instead of stripping port :80 or :443 from baseUrl, that it could test to see if that was the port and if so, baseUrl = common.getHost() ?

or that common.getHostPort strip the port?

instead of using the getHostPort method and replacing the ports, adding another method called getHost() that returns without ports. my thoughts are mostly around naming, it's slightly confusing to use a method with get ports and then remove them

But I'd still have to use .getHostPort() to know if I need the .getHost() or not.

Or I could use some other method to get the kibana port first from the config, and then either .getHostPort() or .getHost().

So either this;

baseUrl = PageObjects.common.getHostPort(); if (baseUrl.includes(':80') || baseUrl.includes(':443') { baseUrl = PageObjects.common.getHost(); }

or

if (config.kibana.port === '443' || config.kibana.port === '80') { baseUrl = PageObjects.common.getHost(); } else { baseUrl = PageObjects.common.getHostPort(); }

or, set the port to null or an empty string. Maybe it wouldn't be appended then?

ah okay I understand now, this lgtm.

LeeDr · 2016-10-31T22:23:29Z

@jbudz yes, I created this PR on the 5.0 branch because that's what's running on Elastic Cloud and we've already made changes in the tests in master which are not compatible with 5.0.
For example, a test-data-subj was added to the timepicker and the page object was changed to use that test-data-subj, so it won't work against a 5.0 instance of Kibana now.

But I will forward-port it to master.

Backports PR #8880 **Commit 1:** Support Cloud Kibana testing * Original sha: c877b0d * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-28T19:16:31Z **Commit 2:** Merge branch '5.0' into supportCloudTesting * Original sha: 0c2815b * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-28T20:53:48Z **Commit 3:** Add xpack file * Original sha: d095dac * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-29T00:02:18Z **Commit 4:** Use kibana for kibana server user, Don't log password * Original sha: 9f63224 * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-31T17:09:57Z

* Support Cloud Kibana UI testing master * Add xpack file * cherry-pick 9f63224

Backports PR #8880 **Commit 1:** Support Cloud Kibana testing * Original sha: c877b0d * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-28T19:16:31Z **Commit 2:** Merge branch '5.0' into supportCloudTesting * Original sha: 0c2815b * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-28T20:53:48Z **Commit 3:** Add xpack file * Original sha: d095dac * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-29T00:02:18Z **Commit 4:** Use kibana for kibana server user, Don't log password * Original sha: 9f63224 * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-31T17:09:57Z

Backports PR #8915 **Commit 1:** Support Cloud Kibana UI testing master * Original sha: 98ba2ce * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-28T19:16:31Z **Commit 2:** Add xpack file * Original sha: adc27ba * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-29T00:02:18Z **Commit 3:** cherry-pick 9f63224 * Original sha: f4549ee * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-31T17:09:57Z

* Support Cloud Kibana UI testing master * Add xpack file * cherry-pick 9f63224

Backports PR elastic#8915 **Commit 1:** Support Cloud Kibana UI testing master * Original sha: 98ba2ce * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-28T19:16:31Z **Commit 2:** Add xpack file * Original sha: adc27ba * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-29T00:02:18Z **Commit 3:** cherry-pick 9f63224 * Original sha: f4549ee * Authored by LeeDr <lee.drengenberg@elastic.co> on 2016-10-31T17:09:57Z Former-commit-id: 9eb4f5b

Support Cloud Kibana testing

c877b0d

LeeDr added the test label Oct 28, 2016

LeeDr assigned rashmivkulkarni and jbudz Oct 28, 2016

Merge branch '5.0' into supportCloudTesting

0c2815b

LeeDr added v5.0.0 v5.0.1 v5.1.0 v6.0.0 labels Oct 28, 2016

Add xpack file

d095dac

jbudz reviewed Oct 31, 2016

View reviewed changes

Use kibana for kibana server user, Don't log password

9f63224

jbudz approved these changes Oct 31, 2016

View reviewed changes

LeeDr merged commit 4a38a3f into elastic:5.0 Nov 1, 2016

elastic-jasper mentioned this pull request Nov 1, 2016

[master] Support Cloud Kibana UI testing #8913

Closed

LeeDr pushed a commit that referenced this pull request Nov 1, 2016

Port #8880 to master Support Cloud testing (#8915)

8ed3b33

* Support Cloud Kibana UI testing master * Add xpack file * cherry-pick 9f63224

elastic-jasper mentioned this pull request Nov 1, 2016

[5.x] Support Cloud Kibana UI testing #8916

Closed

elastic-jasper mentioned this pull request Nov 1, 2016

[5.x] Port #8880 to master Support Cloud testing #8917

Merged

LeeDr removed v5.1.0 v6.0.0 labels Nov 1, 2016

nreese pushed a commit to nreese/kibana that referenced this pull request Nov 10, 2016

Port elastic#8880 to master Support Cloud testing (elastic#8915)

3b10171

* Support Cloud Kibana UI testing master * Add xpack file * cherry-pick 9f63224

LeeDr deleted the supportCloudTesting branch August 20, 2020 17:52

		@@ -20,6 +20,11 @@ bdd.describe('shared links', function describeIndexTests() {

		bdd.before(function () {
		baseUrl = PageObjects.common.getHostPort();

Conversation

LeeDr commented Oct 28, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LeeDr commented Oct 28, 2016

Uh oh!

rashmivkulkarni commented Oct 29, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jbudz commented Oct 31, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LeeDr commented Oct 31, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Comments

LeeDr commented Oct 28, 2016 •

edited

Loading

LeeDr commented Oct 31, 2016 •

edited

Loading