Fix cleanup in multicluster tests#909
Conversation
The cleanup effectively ignored the `KEEP_ON_FAILURE` flag since `CurrentSpecReport()` at this level doesn't return any usable information since all the tests have finished. Instead, we use `ReportAfterSuite` which recieves a concentrated report and can tell if the suite has passed or failed. Signed-off-by: Mike Kolesnik <mkolesni@redhat.com>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #909 +/- ##
==========================================
- Coverage 76.42% 76.27% -0.15%
==========================================
Files 44 44
Lines 2685 2685
==========================================
- Hits 2052 2048 -4
- Misses 541 544 +3
- Partials 92 93 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
|
||
| var _ = AfterSuite(func(ctx SpecContext) { | ||
| if CurrentSpecReport().Failed() { | ||
| var _ = ReportAfterSuite("Condiotnal cleanup", func(ctx SpecContext, r Report) { |
There was a problem hiding this comment.
Pushed the following PR to fix the small typo: #911
* upstream/main: (25 commits) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) Add to vendors_default the possibility to add values for IstioCNI (istio-ecosystem#845) Updates release and nightly-images GH workflows to work with new FBC used in community-operators-prod (istio-ecosystem#854) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (27 commits) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) Docs: adding istiod HA documentations (istio-ecosystem#865) ...
* upstream/main: (28 commits) Fix docs for "broken_links" workflow (istio-ecosystem#918) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) Adding description for release workflow run (istio-ecosystem#868) ...
* upstream/main: (29 commits) e2e allow to build and push setting target arch (istio-ecosystem#913) Fix docs for "broken_links" workflow (istio-ecosystem#918) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) Split Documentation: splitting docs, adding table of content and fixing links (istio-ecosystem#881) Do not block the workflow on errors when publishing to operatorhub.io (istio-ecosystem#877) chore(docs): cleans up IstioCNI section (istio-ecosystem#867) Force push to operator hub fork (istio-ecosystem#873) Make a yaml linter in the community-operators-prod happy (istio-ecosystem#869) ...
* upstream/main: (34 commits) Update VERSIONS_YAML_FILE e2e test information (istio-ecosystem#916) Improve automated docs test run (istio-ecosystem#894) Use `Eventually` in remote secret generation (istio-ecosystem#921) Docs followup link fix (istio-ecosystem#919) Fix helm issue during e2e test run operator installation (istio-ecosystem#914) e2e allow to build and push setting target arch (istio-ecosystem#913) Fix docs for "broken_links" workflow (istio-ecosystem#918) Revert "Disable imageDigest support in ztunnel controller (openshift-service-mesh#582)" (istio-ecosystem#905) Fix typo in the ReportAfterSuite text (istio-ecosystem#911) Fix cleanup in multicluster tests (istio-ecosystem#909) Adding Eventually to Helm operator install (istio-ecosystem#907) Skip cleanup if control plane tests fail (istio-ecosystem#910) Use debug logging in integration tests (istio-ecosystem#906) Annotate only our clusters when using kind (istio-ecosystem#904) Fix log message for cleaner when waiting (istio-ecosystem#903) Consolidate multicluster operator deploy/cleanup (istio-ecosystem#902) Improve e2e test cleanup by recording & cleaning up (istio-ecosystem#889) Enchance debug information when e2e fails: add istioctl proxy status (istio-ecosystem#891) Update github action for broken links (istio-ecosystem#888) Fix "Check broken link" workflow (istio-ecosystem#887) ...
The cleanup effectively ignored the `KEEP_ON_FAILURE` flag since `CurrentSpecReport()` at this level doesn't return any usable information since all the tests have finished. Instead, we use `ReportAfterSuite` which recieves a concentrated report and can tell if the suite has passed or failed. Signed-off-by: Mike Kolesnik <mkolesni@redhat.com>
The cleanup effectively ignored the `KEEP_ON_FAILURE` flag since `CurrentSpecReport()` at this level doesn't return any usable information since all the tests have finished. Instead, we use `ReportAfterSuite` which recieves a concentrated report and can tell if the suite has passed or failed. Signed-off-by: Mike Kolesnik <mkolesni@redhat.com>
|
/cherry-pick release-1.26 |
|
@dgn: #909 failed to apply on top of branch "release-1.26": DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
@dgn: new issue created for failed cherrypick: #980 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
The cleanup effectively ignored the `KEEP_ON_FAILURE` flag since `CurrentSpecReport()` at this level doesn't return any usable information since all the tests have finished. Instead, we use `ReportAfterSuite` which recieves a concentrated report and can tell if the suite has passed or failed. Signed-off-by: Mike Kolesnik <mkolesni@redhat.com> Signed-off-by: Daniel Grimm <dgrimm@redhat.com>
The cleanup effectively ignored the
KEEP_ON_FAILUREflag sinceCurrentSpecReport()at this level doesn't return any usable information since all the tests have finished.Instead, we use
ReportAfterSuitewhich recieves a concentrated report and can tell if the suite has passed or failed.What type of PR is this?
What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Related Issue/PR #
Additional information: