Bump bwc on main with 3.3.2 bwcVersions #19734

peterzhuamazon · 2025-10-22T19:01:59Z

Pending 3.3.2 official snapshots/min being built before merging

Description

Bump bwc on main with 3.3.2 bwcVersions

Related Issues

opensearch-project/opensearch-build#5784

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2025-10-22T19:48:03Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

peterzhuamazon · 2025-10-22T21:49:03Z

New min artifacts is building now.

github-actions · 2025-10-22T23:44:14Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

jainankitk

Let us merge this in asap as it is causing gradle check to fail on main with below error:

Execution failed for task ':distribution:bwc:bugfix:buildBwcLinuxTar'.
> Building 3.3.1 didn't generate expected file /Users/akjain/ws/OpenSearch/distribution/bwc/bugfix/build/bwc/checkout-3.3/distribution/archives/linux-tar/build/distributions/opensearch-min-3.3.1-SNAPSHOT-linux-x64.tar.gz

github-actions · 2025-10-23T02:32:42Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

jainankitk · 2025-10-23T02:57:47Z

Seems like a flaky test:

[Test Result](https://build.ci.opensearch.org/job/gradle-check/66006/testReport/) (1 failure / ±0)

    [org.opensearch.smoketest.SmokeTestMultiNodeClientYamlTestSuiteIT.test {yaml=search/95_search_after_shard_doc/accept _shard_doc with PIT + paginate with search_after (multi-shard)}](https://build.ci.opensearch.org/job/gradle-check/66006/testReport/junit/org.opensearch.smoketest/SmokeTestMultiNodeClientYamlTestSuiteIT/test__yaml_search_95_search_after_shard_doc_accept__shard_doc_with_PIT___paginate_with_search_after__multi_shard__/)

github-actions · 2025-10-23T03:39:13Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-10-23T04:25:10Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-10-23T05:28:15Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-10-23T06:10:09Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

sandeshkr419 · 2025-10-23T06:13:48Z

* What went wrong:
Execution failed for task ':distribution:bwc:bugfix:buildBwcLinuxTar'.
> Building 3.3.2 didn't generate expected file /var/jenkins/workspace/gradle-check/search/distribution/bwc/bugfix/build/bwc/checkout-3.3/distribution/archives/linux-tar/build/distributions/opensearch-min-3.3.2-SNAPSHOT-linux-x64.tar.gz

jainankitk · 2025-10-23T06:20:45Z

The build is passing for me locally with this change:

 [3.3.2] BUILD SUCCESSFUL in 14s
 [3.3.2] 223 actionable tasks: 5 executed, 218 up-to-date

[Incubating] Problems report is available at: file:///Users/akjain/ws/OpenSearch/build/reports/problems/problems-report.html

BUILD SUCCESSFUL in 18s
12 actionable tasks: 4 executed, 8 up-to-date

and I can see that the file is present:

% ls distribution/bwc/bugfix/build/bwc/checkout-3.3/distribution/archives/linux-tar/build/distributions/
opensearch-min-3.3.2-SNAPSHOT-linux-x64.tar.gz

but jenkins build is not passing for some reason

github-actions · 2025-10-23T06:50:03Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-10-23T09:35:12Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

peterzhuamazon · 2025-10-23T17:36:10Z

Both snapshot jars and min tar is available already, let's see in the next gradle check run.

github-actions · 2025-10-23T17:40:33Z

❌ Gradle check result for 0dac7cf: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

jainankitk · 2025-10-23T17:45:03Z

Both snapshot jars and min tar is available already, let's see in the next gradle check run.

@peterzhuamazon - Still seeing the error related to jars and min tar, not sure what's going on here:

* What went wrong:
Execution failed for task ':distribution:bwc:bugfix:buildBwcLinuxTar'.
> Building 3.3.2 didn't generate expected file /var/jenkins/workspace/gradle-check/search/distribution/bwc/bugfix/build/bwc/checkout-3.3/distribution/archives/linux-tar/build/distributions/opensearch-min-3.3.2-SNAPSHOT-linux-x64.tar.gz

Signed-off-by: Peter Zhu <[email protected]>

andrross · 2025-10-23T18:06:45Z

> Task :distribution:bwc:bugfix:checkoutBwcBranch
Commit date of current: 'Wed, 22 Oct 2025 15:01:02 -0400'
Performing checkout of opensearch-project/3.3...
Checkout hash for :distribution:bwc:bugfix is d86afb704dc5cb46773206c77b79743658c88e9f

@jainankitk I believe the bwc tests will attempt to align based on date with a commit on the bwc branch. This prevents non-rebased PRs from failing due to things like serialization changes on newer PRs that are merge ahead of them. In this case we were missing the commit on the 3.3 branch that bumped the version to 3.3.2. I've rebased this PR which should fix it.

jainankitk · 2025-10-23T18:47:24Z

@jainankitk I believe the bwc tests will attempt to align based on date with a commit on the bwc branch. This prevents non-rebased PRs from failing due to things like serialization changes on newer PRs that are merge ahead of them.

Thanks @andrross for the explanation. This makes sense and answers few questions I have been wondering about regarding this version bump process.

In this case we were missing the commit on the 3.3 branch that bumped the version to 3.3.2. I've rebased this PR which should fix it.

Are you saying the 3.3 branch HEAD for bwc tests on this PR was set to something before #19732. But, that should not be the case if this PR was created after that was merged in!?

github-actions · 2025-10-23T19:16:28Z

❌ Gradle check result for b028e7a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

jainankitk · 2025-10-23T19:23:25Z

Few more flaky test failures:

[Test Result](https://build.ci.opensearch.org/job/gradle-check/66066/testReport/) (3 failures / +2)

    [org.opensearch.repositories.azure.AzureBlobStoreRepositoryTests.testSnapshotWithLargeSegmentFiles](https://build.ci.opensearch.org/job/gradle-check/66066/testReport/junit/org.opensearch.repositories.azure/AzureBlobStoreRepositoryTests/testSnapshotWithLargeSegmentFiles/)
    [org.opensearch.remotestore.RemoteRestoreSnapshotIT.testClusterManagerFailoverDuringSnapshotCreation {p0={"opensearch.experimental.feature.writable_warm_index.enabled":"false"}}](https://build.ci.opensearch.org/job/gradle-check/66066/testReport/junit/org.opensearch.remotestore/RemoteRestoreSnapshotIT/testClusterManagerFailoverDuringSnapshotCreation__p0___opensearch_experimental_feature_writable_warm_index_enabled___false___/)
    [org.opensearch.remotestore.RemoteRestoreSnapshotIT.classMethod](https://build.ci.opensearch.org/job/gradle-check/66066/testReport/junit/org.opensearch.remotestore/RemoteRestoreSnapshotIT/classMethod/)

github-actions · 2025-10-23T20:40:58Z

❌ Gradle check result for b028e7a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-10-23T21:32:33Z

❌ Gradle check result for b028e7a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-10-23T22:02:12Z

❌ Gradle check result for b028e7a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

gaiksaya · 2025-10-23T22:10:11Z

Hi folks,
Can you please try to resolve in local first? Looks like consistent failure triggered an alarm on jenkins.

andrross · 2025-10-23T22:26:13Z

@gaiksaya We've resolved the issues on this PR, but we're getting a bit unlucky with flaky failures. However, all other PRs will fail with bwc failures until this one is merged.

andrross · 2025-10-23T22:30:58Z

Are you saying the 3.3 branch HEAD for bwc tests on this PR was set to something before #19732. But, that should not be the case if this PR was created after that was merged in!?

@jainankitk This PR was created at 12:01 PDT. #19732 was merged at 13:14 PDT.

github-actions · 2025-10-23T23:17:07Z

✅ Gradle check result for b028e7a: SUCCESS

codecov · 2025-10-23T23:18:13Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.09%. Comparing base (c6236c3) to head (b028e7a).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #19734      +/-   ##
============================================
- Coverage     73.14%   73.09%   -0.06%     
- Complexity    70797    70879      +82     
============================================
  Files          5732     5735       +3     
  Lines        324258   324654     +396     
  Branches      46925    46962      +37     
============================================
+ Hits         237186   237302     +116     
- Misses        67897    68208     +311     
+ Partials      19175    19144      -31

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Peter Zhu <[email protected]>

peterzhuamazon self-assigned this Oct 22, 2025

peterzhuamazon requested a review from a team as a code owner October 22, 2025 19:02

peterzhuamazon added release skip-changelog labels Oct 22, 2025

peterzhuamazon added this to Engineering Effectiveness Board and OpenSearch Engineering Effectiveness Oct 22, 2025

peterzhuamazon added the v3.3.2 label Oct 22, 2025

github-project-automation bot moved this to Backlog in OpenSearch Engineering Effectiveness Oct 22, 2025

peterzhuamazon moved this to 👀 In Review in Engineering Effectiveness Board Oct 22, 2025

peterzhuamazon moved this from Backlog to In review in OpenSearch Engineering Effectiveness Oct 22, 2025

peterzhuamazon mentioned this pull request Oct 22, 2025

[RELEASE] Release version 3.3.1 opensearch-project/opensearch-build#5784

Closed

65 tasks

cwperks approved these changes Oct 22, 2025

View reviewed changes

peterzhuamazon mentioned this pull request Oct 23, 2025

[RELEASE] Release version 3.3.2 opensearch-project/opensearch-build#5799

Closed

65 tasks

jainankitk approved these changes Oct 23, 2025

View reviewed changes

opensearch-ci-bot mentioned this pull request Oct 23, 2025

[AUTOCUT] Gradle Check Flaky Test Report for SmokeTestMultiNodeClientYamlTestSuiteIT #14408

Open

This was referenced Oct 23, 2025

[AUTOCUT] Gradle Check Flaky Test Report for ResourceAwareTasksTests #14293

Open

[AUTOCUT] Gradle Check Flaky Test Report for FlightMetricsTests #18947

Open

Bump bwc on main with 3.3.2 bwcVersions

b028e7a

Signed-off-by: Peter Zhu <[email protected]>

andrross force-pushed the add-3.3.1-bwc branch from 0dac7cf to b028e7a Compare October 23, 2025 18:03

This was referenced Oct 23, 2025

[AUTOCUT] Gradle Check Flaky Test Report for RemoteStoreReplicationSourceTests #16683

Open

[AUTOCUT] Gradle Check Flaky Test Report for RemoteStoreRefreshListenerTests #17174

Open

sandeshkr419 mentioned this pull request Oct 23, 2025

Add Hybrid Cardinality collector to prioritize Ordinals Collector #19524

Open

msfroh merged commit 0c89456 into opensearch-project:main Oct 23, 2025
37 of 45 checks passed

github-project-automation bot moved this from In review to Done in OpenSearch Engineering Effectiveness Oct 23, 2025

github-project-automation bot moved this from 👀 In Review to ✅ Done in Engineering Effectiveness Board Oct 23, 2025

peterzhuamazon deleted the add-3.3.1-bwc branch October 24, 2025 00:15

karenyrx pushed a commit to karenyrx/OpenSearch that referenced this pull request Oct 24, 2025

Bump bwc on main with 3.3.2 bwcVersions (opensearch-project#19734)

ab13e11

Signed-off-by: Peter Zhu <[email protected]>

lawofcycles pushed a commit to lawofcycles/OpenSearch that referenced this pull request Oct 24, 2025

Bump bwc on main with 3.3.2 bwcVersions (opensearch-project#19734)

3a52e86

Signed-off-by: Peter Zhu <[email protected]>

Bump bwc on main with 3.3.2 bwcVersions #19734

Bump bwc on main with 3.3.2 bwcVersions #19734

Uh oh!

Conversation

peterzhuamazon commented Oct 22, 2025

Pending 3.3.2 official snapshots/min being built before merging

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

peterzhuamazon commented Oct 22, 2025

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

jainankitk left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

jainankitk commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

sandeshkr419 commented Oct 23, 2025

Uh oh!

jainankitk commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

peterzhuamazon commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

jainankitk commented Oct 23, 2025

Uh oh!

andrross commented Oct 23, 2025

Uh oh!

jainankitk commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

jainankitk commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

gaiksaya commented Oct 23, 2025

Uh oh!

andrross commented Oct 23, 2025

Uh oh!

andrross commented Oct 23, 2025

Uh oh!

github-actions bot commented Oct 23, 2025

Uh oh!

codecov bot commented Oct 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

codecov bot commented Oct 23, 2025 •

edited

Loading