Upgrades cypress to 13.15.1 by michaelolo24 · Pull Request #198715 · elastic/kibana

michaelolo24 · 2024-11-01T18:22:32Z

Summary

This PR upgrades cypress to 13.15.1. This is needed to fix an very annoying issue where Cypress tests hangs when ran locally and when the window is not focused

See Cypress issue here: cypress-io/cypress#28392

elasticmachine · 2024-11-01T18:22:35Z

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine · 2024-11-05T21:12:02Z

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

github-actions · 2024-11-05T21:12:11Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

cauemarcondes · 2024-11-07T14:55:41Z

@michaelolo24 The APM cypress tests failing is a known issue and we'll fix that soon. Sorry for that.

michaelolo24 · 2024-11-07T14:57:07Z

@michaelolo24 The APM cypress tests failing is a known issue and we'll fix that soon. Sorry for that.

Thanks @cauemarcondes ! Since I would rather not skip tests, I'll wait for those issues to be fixed and then will push this upgrade. Thank you!

michaelolo24 · 2024-11-07T16:28:34Z

@cauemarcondes - We actually may end up skipping them to get the upgrade in as running cypress locally has become increasingly difficult. I tried to be judicious in skipping only the specific tests that we're failing rather than whole blocks.

If it helps, I think the fix for service_overview, service_inventory, and agent_configuration files may involve either updating the input/combobox components for environmentFilter and serviceNameComboBox to be more easily modifiable or updating the tests to not rely on typing in them

For the integration_polciy and tutorial files, the pages don't seem to be loading the views properly so not sure what's happening there

cauemarcondes · 2024-11-07T16:53:29Z

I've got a PR skipping the problematic tests: #199336

cauemarcondes · 2024-11-08T10:54:36Z

@michaelolo24 my PR has been merged.

PhilippeOberti · 2024-11-12T16:39:51Z

@michaelolo24 my PR has been merged.

thanks! I rebased this PR accordingly

PhilippeOberti · 2024-11-12T22:24:12Z

@cauemarcondes can you take one more look and review this PR? The build is green it's ready to go and we need an approval from the @elastic/obs-ux-infra_services-team.
Thanks!

cauemarcondes · 2024-11-13T10:37:15Z

...ins/observability_solution/apm/ftr_e2e/cypress/e2e/service_inventory/service_inventory.cy.ts

@michaelolo24 these tests are not failing on main AFAICT

I unskipped all the tests, let's see what the build says!

@cauemarcondes over the last 2 builds (this one and that one) we have 3 tests failing:

service_inventory/service_inventory.cy.ts

service_overview/service_overview.cy.ts

settings/agent_configurations.cy.ts

These are the 3 tests that @michaelolo24 had skipped and I unskipped in my last commit.
If these are not failing on main, can you help us figuring out why these 3 tests would consistently fail?

@PhilippeOberti / @michaelolo24 after checking out your branch and running the test I could not see any specific problem with the tests you mentioned. They are breaking, as CI pointed, but there is no clear reason why. This is the error showed on the console:

2024-11-18 11:58:49.047 Cypress Helper (Renderer)[30120:5226514] CoreText note: Set a breakpoint on CTFontLogSystemFontNameRequest to debug. We detected that the Electron Renderer process just crashed. We have failed the current spec but will continue running the next spec. This can happen for a number of different reasons. If you're running lots of tests on a memory intense application. - Try increasing the CPU/memory on the machine you're running on. - Try enabling experimentalMemoryManagement in your config file. - Try lowering numTestsKeptInMemory in your config file during 'cypress open'. You can learn more here: https://on.cypress.io/renderer-process-crashed (Results) ┌────────────────────────────────────────────────────────────────────────────────────────────────┐ │ Tests: 1 │ │ Passing: 0 │ │ Failing: 1 │ │ Pending: 0 │ │ Skipped: 0 │ │ Screenshots: 0 │ │ Video: true │ │ Duration: 11 seconds │ │ Spec Ran: service_inventory.cy.ts │ └────────────────────────────────────────────────────────────────────────────────────────────────┘

I double-cheked and the tests are indeed passing on the main branch.

I remember @michaelolo24 added this experimentalMemoryManagement to the config file (here) but I think Glo said it wasn't in the right place.
I just pushed a commit moving it to the e2e location, hopefully that will do the trick here... if not I don't know what to do...

considering there are only 3 files modified in this PR, 2 of them are package.json and its lock file, and the third one being the experimentalMemoryManagement change to the config file to try to fix the failing tests, I think it's safe to say that the failures are due to the Cypress update...
If we want to continue updating Cypress, I think we need to re-work these 3 tests a bit to make them work with the newer version.

This version works. Although we're not doing like the previous test, we are just opening the combo box and selecting the element.

it('with the correct environment when changing the environment', () => { cy.intercept('GET', '/internal/apm/suggestions?*').as('environmentSuggestionOptions'); cy.wait(mainAliasNames); cy.wait('@environmentSuggestionOptions'); cy.getByTestSubj('environmentFilter').click(); cy.contains('button', 'production').click(); cy.expectAPIsToHaveBeenCalledWith({ apisIntercepted: mainAliasNames, value: 'environment=production', }); });

Thanks @cauemarcondes, makes sense ! Would you be willing to commit the changes for the three files directly to the PR?

Hey @michaelolo24 I was trying to fix the tests but the browser keeps crasing all the time with this message:

We detected that the Chrome Renderer process just crashed. We have failed the current spec but will continue running the next spec. This can happen for a number of different reasons. If you're running lots of tests on a memory intense application. - Try increasing the CPU/memory on the machine you're running on. - Try enabling experimentalMemoryManagement in your config file. - Try lowering numTestsKeptInMemory in your config file during 'cypress open'. You can learn more here: https://on.cypress.io/renderer-process-crashed

Unfortunatly I don't feel comfortable approving this PR until this is fixed.

Yea, this goes back to the error I was seeing when interacting with the environmentFilter dropdown. Suspecting cypress crashes due to an infinite loop of trying to delete the field values, but not being able to delete...but not 100% sure. I was able to get the changes I made working locally, but because of the way the component works, it's hard to properly test the suggestion api since you can't clear the environment field. We'll see if this current ci run passes. If so, please feel free to make modifications you'd like to make to my changes to better fit APM's expectations as the tests shouldn't crash any longer 🤞🏾

elasticmachine · 2024-11-26T02:00:50Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 117565d
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-198715-117565d9c729

Failed CI Steps

Jest Tests #19

Test Failures

[job] [logs] Jest Tests #19 / Category can submit without setting a category

Metrics [docs]

✅ unchanged

History

💚 Build #254225 succeeded 16972f2330b707e84d6e4249c5928b1b807bb3fa
💚 Build #253653 succeeded 68d38eb54f261bd056a81bd2af55feecc50373aa
💔 Build #253623 failed 99ac2ffb97ae7f808a031cdf30b97ce148cd330a
💛 Build #253366 was flaky c87d094ba1fa2020c062e1a62545b998e6e03f28
💔 Build #251976 failed b412cf9269f160f79750e470f7dc9eab159de280

kibanamachine · 2024-11-26T02:32:37Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7480

[✅] APM - Cypress: 25/25 tests passed.

see run history

cauemarcondes

LGTM

## Summary This PR upgrades cypress to `13.15.1`. This is needed to fix an very annoying issue where Cypress tests hangs when ran locally and when the window is not focused See Cypress issue here: cypress-io/cypress#28392 --------- Co-authored-by: PhilippeOberti <philippe.oberti@elastic.co>

michaelolo24 added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. ci:all-cypress-suites v8.17.0 labels Nov 1, 2024

michaelolo24 requested a review from MadameSheema November 1, 2024 18:22

michaelolo24 mentioned this pull request Nov 1, 2024

[Security Solution][Threat Hunting] - Unskip threat hunting tests #195662

Merged

PhilippeOberti approved these changes Nov 1, 2024

View reviewed changes

michaelolo24 force-pushed the upgrade-cypress branch 2 times, most recently from ae5760c to 574336a Compare November 5, 2024 21:11

michaelolo24 requested a review from a team November 5, 2024 21:11

botelastic bot added ci:project-deploy-observability Create an Observability project Team:obs-ux-infra_services - DEPRECATED DEPRECATED - Use Team:obs-presentation. labels Nov 5, 2024

michaelolo24 force-pushed the upgrade-cypress branch 3 times, most recently from 2177bcc to 4c23685 Compare November 7, 2024 00:11

michaelolo24 force-pushed the upgrade-cypress branch from 3f13009 to 3261fd1 Compare November 7, 2024 16:20

michaelolo24 force-pushed the upgrade-cypress branch from 3261fd1 to a7c48ac Compare November 7, 2024 16:37

PhilippeOberti force-pushed the upgrade-cypress branch from a7c48ac to 73fbb46 Compare November 12, 2024 16:39

cauemarcondes requested changes Nov 13, 2024

View reviewed changes

michaelolo24 and others added 11 commits November 25, 2024 19:52

Upgrade cypress

4d26f47

bump cypress patch

3e95d8b

skip known failing apm tests

bc4272a

fix conflict

b95ade6

unskiping apm tests

e5b241b

trying to move experimentalMemoryManagement in the correct location

e488721

address apm tests

c3a135a

fix last cypress test

68c2f77

remove .only

c79022b

ensure options list visibility

ed933a1

chain assertion

117565d

michaelolo24 force-pushed the upgrade-cypress branch from 16972f2 to 117565d Compare November 26, 2024 00:52

michaelolo24 added v9.0.0 v8.18.0 and removed v8.17.0 labels Nov 26, 2024

cauemarcondes approved these changes Nov 26, 2024

View reviewed changes

michaelolo24 merged commit 1865869 into elastic:main Nov 26, 2024

kibanamachine mentioned this pull request Dec 11, 2024

Upgrade mocha to 10.3.0 #203500

Merged

This was referenced Dec 13, 2024

Update dependency @elastic/elasticsearch to ^8.16.0 (main) #200275

Merged

Sustainable Kibana Architecture: Move modules owned by @elastic/obs-knowledge-team #202766

Merged

jbudz mentioned this pull request Jan 3, 2025

[8.x] Upgrade cypress to 13.17.0 (#205437) #205457

Merged

Conversation

michaelolo24 commented Nov 1, 2024 • edited by PhilippeOberti Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

elasticmachine commented Nov 1, 2024

Uh oh!

elasticmachine commented Nov 5, 2024

Uh oh!

github-actions bot commented Nov 5, 2024

🤖 GitHub comments

Uh oh!

cauemarcondes commented Nov 7, 2024

Uh oh!

michaelolo24 commented Nov 7, 2024

Uh oh!

michaelolo24 commented Nov 7, 2024

Uh oh!

cauemarcondes commented Nov 7, 2024

Uh oh!

cauemarcondes commented Nov 8, 2024

Uh oh!

PhilippeOberti commented Nov 12, 2024

Uh oh!

PhilippeOberti commented Nov 12, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PhilippeOberti Nov 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cauemarcondes Nov 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Nov 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

Uh oh!

kibanamachine commented Nov 26, 2024

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7480

Uh oh!

cauemarcondes left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

michaelolo24 commented Nov 1, 2024 •

edited by PhilippeOberti

Loading

PhilippeOberti Nov 14, 2024 •

edited

Loading

cauemarcondes Nov 18, 2024 •

edited

Loading

elasticmachine commented Nov 26, 2024 •

edited

Loading