Fix flaky tests using custom port mapping when running specs in parallel #6763
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Using a global counter currently does not work because Ginkgo runs parallel specs by isolating them in completely different processes [1]. This relies on the parallel process index returned by Ginkgo to get a value that can be used as starting port. Callers can then safely increment this value if needed in the specs to get a different custom value. [1] https://onsi.github.io/ginkgo/#mental-model-how-ginkgo-runs-parallel-specs
✅ Deploy Preview for odo-docusaurus-preview canceled.
|
openshift-ci
bot
added
kind/bug
rm3l
changed the title
|
OpneShift Failure: [odo] I0424 08:21:40.475625 6633 delete.go:94] failed to delete resource "my-k8s-component" (apps.v1.deployments): timeout while waiting for "my-k8s-component" resource to be deleted
[odo] Failed to delete the "Deployment" resource: my-k8s-component
[odo] ✗ Dev mode interrupted by user
Running oc with args [oc get deployments] and odo env: []
[oc] NAME READY UP-TO-DATE AVAILABLE AGE
[oc] my-k8s-component 0/1 0 0 4m49s
[FAILED] in [It] - /go/odo_1/tests/integration/cmd_dev_test.go:1967 @ 04/24/23 08:21:40.91
Deleting project: cmd-dev-test1919kpy
Running oc with args [oc delete project cmd-dev-test1919kpy --wait=false] and odo env: []
[oc] project.project.openshift.io "cmd-dev-test1919kpy" deleted
Setting current dir to: /go/odo_1/tests/integration
Deleting dir: /tmp/1257254853
Deleting dir: /tmp/3210332516
<< Timeline
[FAILED] Expected
<string>: NAME READY UP-TO-DATE AVAILABLE AGE
my-k8s-component 0/1 0 0 4m49s
not to contain substring
<string>: my-k8s-component
In [It] at: /go/odo_1/tests/integration/cmd_dev_test.go:1967 @ 04/24/23 08:21:40.91
------------------------------
[FAILED] Timed out after 180.000s.
Expected
<string>: âš Pod is Terminating
✗ Finished executing the application (command: devrun) [47s]
âš No pod exists
âš Pod is Pending
✓ Pod is Running
• Syncing files into the container ...
✓ Syncing files into the container [972ms]
• Building your application in container (command: devbuild) ...
✓ Building your application in container (command: devbuild) [5s]
• Executing the application (command: devrun) ...
• Waiting for the application to be ready ...
✓ Waiting for the application to be ready [1s]
- Forwarding from 127.0.0.1:31500 -> 3000
↪ Dev mode
Keyboard Commands:
[Ctrl+c] - Exit and delete resources from the cluster
[p] - Manually apply local changes to the application on the cluster
To satisfy at least one of these matchers: [%!s(*matchers.ContainSubstringMatcher=&{Pushing files... []}) %!s(*matchers.ContainSubstringMatcher=&{Updating Component... []})]
In [BeforeEach] at: /go/odo_1/tests/helper/helper_run.go:70 @ 04/24/23 08:27:00.074
Summarizing 2 Failures:
[FAIL] odo dev command tests 1. devfile contains composite apply command when odo dev is running [It] should execute the composite apply commands successfully
/go/odo_1/tests/integration/cmd_dev_test.go:1967
[FAIL] odo dev command tests port-forwarding for the component when devfile has single endpoint when running odo dev when modifying memoryLimit for container in Devfile [BeforeEach] should react on the Devfile modification (podman=false, manual=true, customPortForwarding=true)
/go/odo_1/tests/helper/helper_run.go:70
Ran 452 of 824 Specs in 2061.940 seconds
FAIL! -- 450 Passed | 2 Failed | 0 Pending | 372 Skipped
Windows Failure: Running oc.exe with args [oc new-project cmd-dev-test2640hhg] and odo env: []
[oc] Unable to connect to the server: dial tcp 158.177.79.34:31700: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
[FAILED] in [BeforeEach] - C:/Users/Administrator.ANSIBLE-TEST-VS/3768/tests/helper/helper_cmd_wrapper.go:101 @ 04/24/23 03:51:22.89
Deleting project: cmd-dev-test1732lsm
Running oc.exe with args [oc delete project cmd-dev-test1732lsm --wait=false] and odo env: []
[oc] Error from server (NotFound): namespaces "cmd-dev-test1732lsm" not found
[FAILED] in [AfterEach] - C:/Users/Administrator.ANSIBLE-TEST-VS/3768/tests/helper/helper_cmd_wrapper.go:101 @ 04/24/23 03:51:23.083
<< Timeline
[FAILED] No future change is possible. Bailing out early after 21.147s.
Running oc.exe with args [oc new-project cmd-dev-test2640hhg] and odo env: []
Expected
<int>: 1
to match exit code:
<int>: 0
In [BeforeEach] at: C:/Users/Administrator.ANSIBLE-TEST-VS/3768/tests/helper/helper_cmd_wrapper.go:101 @ 04/24/23 03:51:22.89
There were additional failures detected. To view them in detail run ginkgo -vv
-----
Running oc.exe with args [oc delete project interactive-add-binding-test131tiv --wait=false] and odo env: []
[oc] Unable to connect to the server: dial tcp 158.177.79.34:31700: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
[FAILED] in [AfterEach] - C:/Users/Administrator.ANSIBLE-TEST-VS/3768/tests/helper/helper_cmd_wrapper.go:101 @ 04/24/23 03:37:51.852
<< Timeline
[FAILED] No future change is possible. Bailing out early after 21.173s.
Running oc.exe with args [oc delete project interactive-add-binding-test131tiv --wait=false] and odo env: []
Expected
<int>: 1
to match exit code:
<int>: 0
In [AfterEach] at: C:/Users/Administrator.ANSIBLE-TEST-VS/3768/tests/helper/helper_cmd_wrapper.go:101 @ 04/24/23 03:37:51.852
----
[FAILED] No future change is possible. Bailing out early after 21.147s.
Running oc.exe with args [oc new-project cmd-dev-test2640hhg] and odo env: []
Expected
<int>: 1
to match exit code:
<int>: 0
In [BeforeEach] at: C:/Users/Administrator.ANSIBLE-TEST-VS/3768/tests/helper/helper_cmd_wrapper.go:101 @ 04/24/23 03:51:22.89
+S++++++++++S+SS++S+SS++++++S+++SSS+++++SS+SSS+S+++SS+S+SS++++SS++S+++SSS+SSS+S+++++++++++++++
Summarizing 3 Failures:
[FAIL] odo add binding interactive command tests [AfterEach] when the component is bootstrapped should successfully add binding to the devfile ("lowercase" as naming strategy)
C:/Users/Administrator.ANSIBLE-TEST-VS/3768/tests/helper/helper_cmd_wrapper.go:101
[FAIL] odo dev command tests [BeforeEach] when running odo dev with alternative commands - with metadata.name when running odo dev with a run command should execute the custom non-default run command successfully
C:/Users/Administrator.ANSIBLE-TEST-VS/3768/tests/helper/helper_cmd_wrapper.go:101
[FAIL] odo add binding command tests [BeforeEach] when the component is bootstrapped when no bindable instance is present on the cluster should fail to add binding with no bindable instance found error message
C:/Users/Administrator.ANSIBLE-TEST-VS/3768/tests/helper/helper_generic.go:58
|
valaparthvi
approved these changes
Apr 24, 2023
openshift-ci
bot
added
the
lgtm
Co-authored-by: Parthvi Vala <[email protected]>
openshift-ci
bot
removed
the
lgtm
|
Kudos, SonarCloud Quality Gate passed!
|
|
/override windows-integration-test/Windows-test + [FAILED] [73.488 seconds]
odo generic When deleting two project one after the other [It] should be able to delete sequentially
+ [FAILED] [91.113 seconds]
odo describe/list binding command tests when creating a component with a spec binding and envvars (service in namespace "binding-vup") when Starting a Pg service [BeforeEach] when running dev session should describe the binding
[BeforeEach] C:/Users/Administrator.ANSIBLE-TEST-VS/3790/tests/integration/cmd_describe_list_binding_test.go:652
[It] C:/Users/Administrator.ANSIBLE-TEST-VS/3790/tests/integration/cmd_describe_list_binding_test.go:701
+ [FAILED] [558.467 seconds]
odo describe/list binding command tests when creating a component with a spec binding (service in namespace "") when Starting a Pg service when running dev session [BeforeEach] when changing the current namespace should list the binding with --namespace flag
[BeforeEach] C:/Users/Administrator.ANSIBLE-TEST-VS/3790/tests/integration/cmd_describe_list_binding_test.go:690
+ [FAILED] [270.983 seconds]
odo dev interactive command tests when a component is bootstrapped [It] should sync files when p is pressed
C:/Users/Administrator.ANSIBLE-TEST-VS/3790/tests/integration/interactive_dev_test.go:206/lgtm |
openshift-ci
bot
added
the
lgtm
|
@valaparthvi: Overrode contexts on behalf of valaparthvi: windows-integration-test/Windows-test In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
rm3l
deleted the
6758-flaky-test-port-forwarding-for-the-component-possible-custom-port-collision-when-running-tests-in-parallel
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this:
/kind bug
/area testing
What does this PR do / why we need it:
Using a global counter currently does not work
because Ginkgo runs parallel specs by isolating them in
completely different processes [1].
This relies on the parallel process index returned by Ginkgo
to get a value that can be used as starting port.
[1] https://onsi.github.io/ginkgo/#mental-model-how-ginkgo-runs-parallel-specs
Which issue(s) this PR fixes:
Fixes #6758
PR acceptance criteria:
Unit test
Integration test
Documentation
How to test changes / Special notes to the reviewer: