Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix potential issues with E2E tests #6543

Conversation

rm3l
Copy link
Member

@rm3l rm3l commented Jan 26, 2023

What type of PR is this:
/kind tests
/area testing

What does this PR do / why we need it:
This is an attempt to fix the recurring issues we have been seeing with E2E tests.
In a nutshell, it waits for the application to be actually running in the container before sending a request to the local port created by odo.
Otherwise, sending a request to the local port too early sometimes causes the port forwarding to be automatically restarted (with new random ports because we are running odo dev with --random-ports) (#6013)

Which issue(s) this PR fixes:
Fixes #6463

PR acceptance criteria:

  • Unit test

  • Integration test

  • Documentation

How to test changes / Special notes to the reviewer:

rm3l added 5 commits January 26, 2023 17:33
…ws checking (or not) the remote command execution
…equests to the local forwarded port

Otherwise, sending a request too early causes
"connection refused" errors, which causes port-forwarding to be restarted
(with new local ports because "odo dev" is running with "--random-ports").
As a first start, here we are making sure that the application port is available in the container,
prior to sending requests to the local forwarded port. This should reduce the likeliness
of port-forwarding restarts.

In the future, we might want to make the test detect the new local ports and use those instead.
We could even make sure to make "odo dev" reuse the previous random ports it had, provided
they are available again.
This allows checking the application running in the container.
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Jan 26, 2023
@netlify
Copy link

netlify bot commented Jan 26, 2023

Deploy Preview for odo-docusaurus-preview canceled.

Name Link
🔨 Latest commit e11c63f
🔍 Latest deploy log https://app.netlify.com/sites/odo-docusaurus-preview/deploys/63d3c5b5c8a0630009546927

@openshift-ci openshift-ci bot added kind/tests area/testing Issues or PRs related to testing, Quality Assurance or Quality Engineering labels Jan 26, 2023
@openshift-ci openshift-ci bot requested review from feloy and valaparthvi January 26, 2023 17:14
@rm3l rm3l removed request for feloy and valaparthvi January 26, 2023 17:14
@odo-robot
Copy link

odo-robot bot commented Jan 26, 2023

OpenShift Tests on commit 7c2306e finished successfully.
View logs: TXT HTML

@odo-robot
Copy link

odo-robot bot commented Jan 26, 2023

NoCluster Tests on commit 7c2306e finished successfully.
View logs: TXT HTML

@odo-robot
Copy link

odo-robot bot commented Jan 26, 2023

Unit Tests on commit 7c2306e finished successfully.
View logs: TXT HTML

@odo-robot
Copy link

odo-robot bot commented Jan 26, 2023

Kubernetes Tests on commit 7c2306e finished successfully.
View logs: TXT HTML

@odo-robot
Copy link

odo-robot bot commented Jan 26, 2023

Validate Tests on commit 7c2306e finished successfully.
View logs: TXT HTML

@odo-robot
Copy link

odo-robot bot commented Jan 26, 2023

Windows Tests (OCP) on commit 7c2306e finished with errors.
View logs: TXT HTML

@rm3l
Copy link
Member Author

rm3l commented Jan 26, 2023

All relevant tests passed on all platforms. Let's trigger them one more time.

@rm3l rm3l closed this Jan 26, 2023
@rm3l rm3l reopened this Jan 26, 2023
@rm3l rm3l changed the title [WIP] Fix potential issues with E2E tests Fix potential issues with E2E tests Jan 26, 2023
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. Required by Prow. label Jan 26, 2023
Copy link
Contributor

@feloy feloy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing these tests

tests/helper/helper_generic.go Outdated Show resolved Hide resolved
tests/helper/component_interface.go Outdated Show resolved Hide resolved
tests/e2escenarios/e2e_test.go Outdated Show resolved Hide resolved
rm3l added 2 commits January 27, 2023 13:02
…at allows checking (or not) the remote command execution
@rm3l rm3l requested a review from feloy January 27, 2023 12:40
@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. Required by Prow. label Jan 27, 2023
@feloy feloy closed this Jan 27, 2023
@feloy feloy reopened this Jan 27, 2023
@sonarqubecloud
Copy link

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@feloy
Copy link
Contributor

feloy commented Jan 27, 2023

/override Kubernetes-Integration-Tests/Kubernetes-Docs-Integration-Tests
/override windows-integration-test/Windows-test
Known flaky test on logs

@openshift-ci
Copy link

openshift-ci bot commented Jan 27, 2023

@feloy: Overrode contexts on behalf of feloy: Kubernetes-Integration-Tests/Kubernetes-Docs-Integration-Tests, windows-integration-test/Windows-test

In response to this:

/override Kubernetes-Integration-Tests/Kubernetes-Docs-Integration-Tests
/override windows-integration-test/Windows-test
Known flaky test on logs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@feloy
Copy link
Contributor

feloy commented Jan 27, 2023

/override ci/prow/v4.11-integration-e2e
Previously passed

@openshift-ci
Copy link

openshift-ci bot commented Jan 27, 2023

@feloy: Overrode contexts on behalf of feloy: ci/prow/v4.11-integration-e2e

In response to this:

/override ci/prow/v4.11-integration-e2e
Previously passed

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@feloy
Copy link
Contributor

feloy commented Jan 27, 2023

/override ci/prow/v4.11-integration-e2e
Previously passed

@openshift-ci
Copy link

openshift-ci bot commented Jan 27, 2023

@feloy: Overrode contexts on behalf of feloy: ci/prow/v4.11-integration-e2e

In response to this:

/override ci/prow/v4.11-integration-e2e
Previously passed

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@feloy
Copy link
Contributor

feloy commented Jan 27, 2023

/override ci/prow/unit-and-validate-test
Previously passed

@openshift-ci
Copy link

openshift-ci bot commented Jan 27, 2023

@feloy: Overrode contexts on behalf of feloy: ci/prow/unit-and-validate-test

In response to this:

/override ci/prow/unit-and-validate-test
Previously passed

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-merge-robot openshift-merge-robot merged commit 34ecccd into redhat-developer:main Jan 27, 2023
@rm3l rm3l deleted the 6463-recurring-issue-with-e2e-tests branch January 28, 2023 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/testing Issues or PRs related to testing, Quality Assurance or Quality Engineering lgtm Indicates that a PR is ready to be merged. Required by Prow.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Recurring issue with E2E tests
3 participants