Skip to content

Comments

[Canvas] Fix flaky custom element functional tests#65908

Merged
cqliu1 merged 4 commits intoelastic:masterfrom
cqliu1:fix/custom-elements-tests
May 19, 2020
Merged

[Canvas] Fix flaky custom element functional tests#65908
cqliu1 merged 4 commits intoelastic:masterfrom
cqliu1:fix/custom-elements-tests

Conversation

@cqliu1
Copy link
Contributor

@cqliu1 cqliu1 commented May 8, 2020 •
edited
Loading

Summary

Closes #62927.
Closes #63004.
Related to #63042.
Related to #63339.

The functional tests for creating a new custom element and editing an existing custom element would occasionally fail because the test runner sometimes fails to enter the value for the Name input field, possibly from executing too quickly.

This adds the clearWithKeyboard option when setting the value on the inputs in the custom element form which seems to reliably enter both the name and description values on the custom element.

I ran the flaky test runner with these changes 100 times, and all of these functional tests passed.

Results: https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/437/

It's unclear if this changes will fix the delete custom element test failures, but so far, it looks like all custom element tests are passing consistently.

This also adds the path to the Canvas functional tests in .github/CODEOWNERS so the team gets pinged on functional test changes.

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@cqliu1 cqliu1 added review Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas t// loe:medium Medium Level of Effort v8.0.0 release_note:skip Skip the PR/issue when compiling release notes impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. v7.8.0 v7.6.3 v7.7.1 labels May 8, 2020
@elasticmachine
Copy link
Contributor

elasticmachine commented May 8, 2020

Pinging @elastic/kibana-canvas (Team:Canvas)

@cqliu1 cqliu1 requested a review from poffdeluxe May 12, 2020 16:31
@cqliu1 cqliu1 force-pushed the fix/custom-elements-tests branch from 7af94b4 to 66f3136 Compare May 12, 2020 16:34
@cqliu1 cqliu1 requested review from a team and removed request for a team May 12, 2020 16:36
@cqliu1 cqliu1 force-pushed the fix/custom-elements-tests branch from 2173d1e to c99a837 Compare May 12, 2020 18:17
@cqliu1 cqliu1 requested a review from a team May 12, 2020 18:19
clintandrewhall
clintandrewhall approved these changes May 13, 2020
View reviewed changes
Copy link
Contributor

@clintandrewhall clintandrewhall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find. Well done!

As we discussed yesterday, if this backports cleanly, let's consider it.

@cqliu1 cqliu1 force-pushed the fix/custom-elements-tests branch from c99a837 to fc551e9 Compare May 14, 2020 17:43
@cqliu1
Copy link
Contributor Author

cqliu1 commented May 15, 2020

@elasticmachine merge upstream

@brianseeders
Copy link
Contributor

brianseeders commented May 15, 2020

This seems like it will need to be backported to 7.x/7.9 as well, it failed today: https://kibana-ci.elastic.co/job/elasticsearch+snapshots+verify/726/execution/node/320/log/?consoleFull

@cqliu1
Copy link
Contributor Author

cqliu1 commented May 18, 2020

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

kibanamachine commented May 18, 2020

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@cqliu1 cqliu1 merged commit 9c11243 into elastic:master May 19, 2020
@cqliu1 cqliu1 deleted the fix/custom-elements-tests branch May 19, 2020 07:00
gmmorris added a commit to gmmorris/kibana that referenced this pull request May 19, 2020
@gmmorris
Merge branch 'master' into saved-objects/scoped-client-with-extra-types
cc95823 
* master: (24 commits)
  [APM] agent config 'profiling_inferred_spans_min_duration' default value is '0ms' but the min value in the field is '1ms' (elastic#66886)
  [Canvas] Fix flaky custom element functional tests (elastic#65908)
  Fix IE specific flexbox min-height issue (elastic#66555)
  [Discover] Unskip doc link functional test (elastic#66884)
  Index pattern management to Kibana platform (elastic#65026)
  Warning and link to support matrix for IE11 (elastic#66512)
  [Reporting] Consolidate Server Type Defs, move some out of Legacy (elastic#66144)
  [SIEM] [Maps] Fixes Network Map empty tooltip (elastic#66828)
  [Endpoint] Encode the index of the alert in the id response (elastic#66919)
  [services/testSubjects] reduce retry usage, add waitForEnabled (elastic#66538)
  [DOCS] Identifies cloud settings for APM (elastic#66935)
  [SIEM][CASE] Fix configuration's page user experience (elastic#66029)
  Resolver: Display node 75% view submenus (elastic#64121)
  [SIEM] Cases] Capture timeline click and open timeline in case view (elastic#66327)
  [APM] Lowercase agent names so icons work (elastic#66824)
  [dev/cli] add support for --no-cache (elastic#66837)
  [Ingest Manager] Better handling of package installation problems (elastic#66541)
  [ML] Enhances api docs for modules endpoints (elastic#66738)
  dont hide errors (elastic#66764)
  [RFC] Global search API (elastic#64284)
  ...
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 21, 2020
@kibanamachine
Copy link
Contributor

kibanamachine commented May 21, 2020

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 65908 or prevent reminders by adding the backport:skip label.

This was referenced May 21, 2020
Merged
Merged
cqliu1 added a commit to cqliu1/kibana that referenced this pull request May 21, 2020
@cqliu1 @elasticmachine
[Canvas] Fix flaky custom element functional tests (elastic#65908)
1e37967 
* Enable clearWithKeyboard option when filling out custom elements form

* Added canvas functional tests path to CODEOWNERS

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@cqliu1 cqliu1 mentioned this pull request May 21, 2020
Merged
cqliu1 added a commit to cqliu1/kibana that referenced this pull request May 21, 2020
@cqliu1 @elasticmachine
[Canvas] Fix flaky custom element functional tests (elastic#65908)
538b3ac 
* Enable clearWithKeyboard option when filling out custom elements form

* Added canvas functional tests path to CODEOWNERS

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
cqliu1 added a commit that referenced this pull request May 21, 2020
@cqliu1 @elasticmachine
[Canvas] Fix flaky custom element functional tests (#65908) (#67192)
6f6b154 
* Enable clearWithKeyboard option when filling out custom elements form

* Added canvas functional tests path to CODEOWNERS

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
cqliu1 added a commit that referenced this pull request May 21, 2020
@cqliu1 @elasticmachine
[Canvas] Fix flaky custom element functional tests (#65908) (#67194)
b03e2bd 
* Enable clearWithKeyboard option when filling out custom elements form

* Added canvas functional tests path to CODEOWNERS

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label May 22, 2020
cqliu1 added a commit that referenced this pull request May 22, 2020
@cqliu1 @elasticmachine
[Canvas] Fix flaky custom element functional tests (#65908) (#67195)
626f2e2 
* Enable clearWithKeyboard option when filling out custom elements form

* Added canvas functional tests path to CODEOWNERS

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@cqliu1 cqliu1 added the v7.9.0 label May 26, 2020
cqliu1 added a commit to cqliu1/kibana that referenced this pull request May 26, 2020
@cqliu1 @elasticmachine
[Canvas] Fix flaky custom element functional tests (elastic#65908)
b2bff27 
* Enable clearWithKeyboard option when filling out custom elements form

* Added canvas functional tests path to CODEOWNERS

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
@cqliu1 cqliu1 mentioned this pull request May 26, 2020
Merged
cqliu1 added a commit that referenced this pull request May 27, 2020
@cqliu1 @elasticmachine
[Canvas] Fix flaky custom element functional tests (#65908) (#67411)
67c46d9 
* Enable clearWithKeyboard option when filling out custom elements form

* Added canvas functional tests path to CODEOWNERS

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@clintandrewhall clintandrewhall clintandrewhall approved these changes

+1 more reviewer

@poffdeluxe poffdeluxe poffdeluxe approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. loe:medium Medium Level of Effort release_note:skip Skip the PR/issue when compiling release notes review Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas t// v7.6.3 v7.7.1 v7.8.0 v7.9.0 v8.0.0

Projects

None yet

Milestone

No milestone

6 participants

@cqliu1 @elasticmachine @brianseeders @kibanamachine @clintandrewhall @poffdeluxe