Skip to content

fix(e2e): Add cleanup trap to prevent false positives in CI test runs#1227

Merged
istio-testing merged 4 commits intoistio-ecosystem:mainfrom
fjglira:issue-1226
Sep 19, 2025
Merged

fix(e2e): Add cleanup trap to prevent false positives in CI test runs#1227
istio-testing merged 4 commits intoistio-ecosystem:mainfrom
fjglira:issue-1226

Conversation

@fjglira
Copy link
Copy Markdown
Contributor

@fjglira fjglira commented Sep 18, 2025

The e2e test script was storing the Ginkgo exit code but could still exit unexpectedly before cleanup, causing CI to mark incomplete test runs as successful (false positives).

What type of PR is this?

  • Enhancement / New Feature
  • Bug Fix
  • Refactor
  • Optimization
  • Test
  • Documentation Update

What this PR does / why we need it:

Problem:
The e2e test script was storing the Ginkgo exit code, but could still exit unexpectedly before cleanup, causing CI to mark incomplete test runs as successful (false positives).

Solution:

  • Added a cleanup() function with trap cleanup EXIT INT TERM to ensure uninstall always runs
  • Moved cleanup logic to the trap so it executes on normal exit, interrupts, and terminations
  • Simplified the main flow to immediately exit with the captured Ginkgo exit code
  • Cleanup errors are suppressed to preserve the original test result

Result:

  • CI now correctly reflects test failures and interruptions
  • JUnit reports are still generated and cleanup always occurs
  • No more false positive test results from incomplete runs

Which issue(s) this PR fixes:

Fixes #1226

Related Issue/PR #

Additional information:

@fjglira fjglira requested a review from a team as a code owner September 18, 2025 20:06
@codecov
Copy link
Copy Markdown

codecov bot commented Sep 18, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 77.54%. Comparing base (749d908) to head (3bbde83).
⚠️ Report is 83 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1227      +/-   ##
==========================================
+ Coverage   77.39%   77.54%   +0.14%     
==========================================
  Files          44       44              
  Lines        2823     2823              
==========================================
+ Hits         2185     2189       +4     
+ Misses        529      526       -3     
+ Partials      109      108       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fjglira fjglira changed the title fix(e2e): Add cleanup trap to prevent false positives in CI test runs WIP fix(e2e): Add cleanup trap to prevent false positives in CI test runs Sep 18, 2025
The e2e test script was storing the Ginkgo exit code but could still exit unexpectedly before cleanup, causing CI to mark incomplete test runs as successful (false positives).

Signed-off-by: Francisco Herrera <fjglira@gmail.com>
TEMPORARY FAILURE: check failure test on CI

Signed-off-by: Francisco Herrera <fjglira@gmail.com>
Fix lint messages

Signed-off-by: Francisco Herrera <fjglira@gmail.com>
This reverts commit 5346be9.

Signed-off-by: Francisco Herrera <fjglira@gmail.com>
@fjglira fjglira changed the title WIP fix(e2e): Add cleanup trap to prevent false positives in CI test runs fix(e2e): Add cleanup trap to prevent false positives in CI test runs Sep 18, 2025
Copy link
Copy Markdown
Contributor

@ctartici ctartici left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@istio-testing istio-testing merged commit e774b44 into istio-ecosystem:main Sep 19, 2025
17 checks passed
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 19, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 19, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 22, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 22, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 22, 2025
* upstream/main:
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 22, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 23, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 24, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Sep 24, 2025
* upstream/main:
  Enhance Documentation: convert markdown file to AsciiDocs (istio-ecosystem#1118)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
@istio-testing
Copy link
Copy Markdown
Collaborator

In response to a cherrypick label: new pull request created: #1232

@FilipB
Copy link
Copy Markdown
Collaborator

FilipB commented Nov 27, 2025

/cherry-pick release-1.26

@istio-testing
Copy link
Copy Markdown
Collaborator

In response to a cherrypick label: #1227 failed to apply on top of branch "release-1.26":

Applying: fix(e2e): Add cleanup trap to prevent false positives in CI test runs
Using index info to reconstruct a base tree...
M	tests/e2e/common-operator-integ-suite.sh
Falling back to patching base and 3-way merge...
Auto-merging tests/e2e/common-operator-integ-suite.sh
CONFLICT (content): Merge conflict in tests/e2e/common-operator-integ-suite.sh
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 fix(e2e): Add cleanup trap to prevent false positives in CI test runs

@istio-testing
Copy link
Copy Markdown
Collaborator

In response to a cherrypick label: new issue created for failed cherrypick: #1406

@istio-testing
Copy link
Copy Markdown
Collaborator

@FilipB: #1227 failed to apply on top of branch "release-1.26":

Applying: fix(e2e): Add cleanup trap to prevent false positives in CI test runs
Using index info to reconstruct a base tree...
M	tests/e2e/common-operator-integ-suite.sh
Falling back to patching base and 3-way merge...
Auto-merging tests/e2e/common-operator-integ-suite.sh
CONFLICT (content): Merge conflict in tests/e2e/common-operator-integ-suite.sh
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 fix(e2e): Add cleanup trap to prevent false positives in CI test runs

Details

In response to this:

/cherry-pick release-1.26

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@istio-testing
Copy link
Copy Markdown
Collaborator

In response to a cherrypick label: #1227 failed to apply on top of branch "release-1.27":

Applying: fix(e2e): Add cleanup trap to prevent false positives in CI test runs
Using index info to reconstruct a base tree...
M	tests/e2e/common-operator-integ-suite.sh
Falling back to patching base and 3-way merge...
Auto-merging tests/e2e/common-operator-integ-suite.sh
CONFLICT (content): Merge conflict in tests/e2e/common-operator-integ-suite.sh
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 fix(e2e): Add cleanup trap to prevent false positives in CI test runs

@istio-testing
Copy link
Copy Markdown
Collaborator

@FilipB: new issue created for failed cherrypick: #1407

Details

In response to this:

/cherry-pick release-1.26

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@istio-testing
Copy link
Copy Markdown
Collaborator

In response to a cherrypick label: new issue created for failed cherrypick: #1408

FilipB pushed a commit to FilipB/sail-operator that referenced this pull request Nov 27, 2025
…istio-ecosystem#1227)

* fix(e2e): Add cleanup trap to prevent false positives in CI test runs

The e2e test script was storing the Ginkgo exit code but could still exit unexpectedly before cleanup, causing CI to mark incomplete test runs as successful (false positives).

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

* TEMPORARY FAILURE

TEMPORARY FAILURE: check failure test on CI

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

* Fix lint

Fix lint messages

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

* Revert "TEMPORARY FAILURE"

This reverts commit 5346be9.

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

---------

Signed-off-by: Francisco Herrera <fjglira@gmail.com>
Signed-off-by: Filip Brychta <fbrychta@redhat.com>
istio-testing pushed a commit that referenced this pull request Dec 2, 2025
…#1227)

* fix(e2e): Add cleanup trap to prevent false positives in CI test runs

The e2e test script was storing the Ginkgo exit code but could still exit unexpectedly before cleanup, causing CI to mark incomplete test runs as successful (false positives).

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

* TEMPORARY FAILURE

TEMPORARY FAILURE: check failure test on CI

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

* Fix lint

Fix lint messages

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

* Revert "TEMPORARY FAILURE"

This reverts commit 5346be9.

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

---------

Signed-off-by: Francisco Herrera <fjglira@gmail.com>
Signed-off-by: Filip Brychta <fbrychta@redhat.com>
openshift-service-mesh-bot pushed a commit to openshift-service-mesh-bot/sail-operator that referenced this pull request Dec 2, 2025
* upstream/release-1.26:
  Make linter happy
  Make sure to skip cleanup of operator in e2e (istio-ecosystem#1351)
  fix(e2e): Add cleanup trap to prevent false positives in CI test runs (istio-ecosystem#1227)
dgn pushed a commit to dgn/sail-operator that referenced this pull request Mar 17, 2026
…istio-ecosystem#1227)

* fix(e2e): Add cleanup trap to prevent false positives in CI test runs

The e2e test script was storing the Ginkgo exit code but could still exit unexpectedly before cleanup, causing CI to mark incomplete test runs as successful (false positives).

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

* TEMPORARY FAILURE

TEMPORARY FAILURE: check failure test on CI

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

* Fix lint

Fix lint messages

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

* Revert "TEMPORARY FAILURE"

This reverts commit 5346be9.

Signed-off-by: Francisco Herrera <fjglira@gmail.com>

---------

Signed-off-by: Francisco Herrera <fjglira@gmail.com>
Signed-off-by: Daniel Grimm <dgrimm@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] Falses positives when the e2e test are not completed and result is not being generated

6 participants