Skip to content

Conversation

@guojialiang92
Copy link
Contributor

Description

This PR is trying to fix a flakey test org.opensearch.common.ssl.PemTrustConfigTests#testTrustConfigReloadsFileContents.

Related Issues

Resolves #[17983]

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

❕ Gradle check result for 0ee096c: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

@codecov
Copy link

codecov bot commented Apr 17, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.51%. Comparing base (cbaddd3) to head (75c5b9f).
⚠️ Report is 364 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##               main   #17984    +/-   ##
==========================================
  Coverage     72.51%   72.51%            
+ Complexity    67108    67101     -7     
==========================================
  Files          5475     5478     +3     
  Lines        309916   310034   +118     
  Branches      45060    45066     +6     
==========================================
+ Hits         224725   224823    +98     
+ Misses        66895    66844    -51     
- Partials      18296    18367    +71     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@gaobinlong gaobinlong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This failed test and another one testBadFileFormatFails always fail with the same AssertionError in my local, could you help to check if it's caused by the dependency upgrade so the error message thrown is changed? If so I think we can change the assertion in assertFailedToParse() directly.

@guojialiang92
Copy link
Contributor Author

guojialiang92 commented Apr 18, 2025

Thanks, @gaobinlong.
You are right! PemTrustConfigTests#testBadFileFormatFails seems to have the same problem.
I reproduced it locally, but it is not stable. About one exception will occur every 100 runs.
测试报错图

I abstracted method PemTrustConfigTests#assertFailedToParseOrInvalidFileFormat and used it for these two flaky tests.

@github-actions
Copy link
Contributor

❌ Gradle check result for 981586d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: guojialiang <[email protected]>
@guojialiang92 guojialiang92 force-pushed the dev/fix-PemTrustConfigTests-testTrustConfigReloadsFileContents branch from 981586d to 37f824e Compare April 18, 2025 14:39
@github-actions
Copy link
Contributor

❌ Gradle check result for 37f824e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for 67d72db: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

❌ Gradle check result for d910aaf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: guojialiang <[email protected]>
@github-actions
Copy link
Contributor

✅ Gradle check result for 75c5b9f: SUCCESS

final SslConfigException exception = expectThrows(SslConfigException.class, trustConfig::createTrustManager);
assertThat(exception.getMessage(), Matchers.containsString(file.toAbsolutePath().toString()));
assertThat(exception.getMessage(), Matchers.containsString("cannot create trust"));
assertThat(exception.getMessage(), Matchers.containsString("PEM"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we merge the two lines 167 and 168 into one?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could combine 166-168 actually.

boolean isFailedToParse = false;
boolean isInvalidFileFormat = false;
try {
assertFailedToParse(trustConfig, file);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know when will Failed to parse happen and when will Invalid format happen? I think it's better to make the trigger condition clear rather than depending on try...catch.

@andrross
Copy link
Member

@guojialiang92 thanks for the PR! Are you still working on this?

@opensearch-trigger-bot
Copy link
Contributor

This PR is stalled because it has been open for 30 days with no activity.

@opensearch-trigger-bot opensearch-trigger-bot bot added stalled Issues that have stalled and removed stalled Issues that have stalled labels Jun 19, 2025
@opensearch-trigger-bot
Copy link
Contributor

This PR is stalled because it has been open for 30 days with no activity.

@opensearch-trigger-bot opensearch-trigger-bot bot added the stalled Issues that have stalled label Aug 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog stalled Issues that have stalled

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants