[Backport 2.x] Compress and cache cluster state during validate join request (#7321) #7945

andrross · 2023-06-06T22:59:52Z

Backports #7321 to 2.x

github-actions · 2023-06-06T23:10:46Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/16987/
CommitID: 91ee919
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-06T23:52:40Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.search.SearchWeightedRoutingIT.testStrictWeightedRoutingWithCustomString

URL: https://build.ci.opensearch.org/job/gradle-check/16989/
CommitID: bcde254
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2023-06-06T23:57:11Z

Codecov Report

Merging #7945 (fb56099) into 2.x (efd12df) will decrease coverage by 0.07%.
The diff coverage is 87.93%.

❗ Current head fb56099 differs from pull request most recent head daeaa52. Consider uploading reports for the commit daeaa52 to get more accurate results

@@             Coverage Diff              @@
##                2.x    #7945      +/-   ##
============================================
- Coverage     70.64%   70.58%   -0.07%     
+ Complexity    56851    56780      -71     
============================================
  Files          4703     4704       +1     
  Lines        269104   269091      -13     
  Branches      39774    39771       -3     
============================================
- Hits         190118   189932     -186     
- Misses        62702    62833     +131     
- Partials      16284    16326      +42

Impacted Files	Coverage Δ
...g/opensearch/cluster/coordination/Coordinator.java	`78.89% <ø> (+3.11%)`	⬆️
...ch/cluster/coordination/CompressedStreamUtils.java	`80.00% <80.00%> (ø)`
...rg/opensearch/cluster/coordination/JoinHelper.java	`84.72% <89.47%> (-0.84%)`	⬇️
...ster/coordination/PublicationTransportHandler.java	`85.18% <100.00%> (-0.68%)`	⬇️

... and 452 files with indirect coverage changes

github-actions · 2023-06-08T18:25:29Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17155/
CommitID: 91f9db9
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

kotwanikunal · 2023-06-13T15:54:03Z

@andrross Can you please rebase and iterate to green?

server/src/main/java/org/opensearch/cluster/coordination/JoinHelper.java

github-actions · 2023-06-15T22:20:03Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/17726/
CommitID: c785e9f
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-16T00:39:59Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testDropPrimaryDuringReplication

URL: https://build.ci.opensearch.org/job/gradle-check/17734/
CommitID: c785e9f
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions · 2023-06-17T00:13:56Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.search.SearchWeightedRoutingIT.testStrictWeightedRoutingWithCustomString
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testPressureServiceStats
      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testDropPrimaryDuringReplication

URL: https://build.ci.opensearch.org/job/gradle-check/17830/
CommitID: fb56099
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <[email protected]>

github-actions · 2023-06-21T15:56:54Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/18092/
CommitID: 0dfe96e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-21T23:49:32Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/18130/
CommitID: 0dfe96e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

andrross · 2023-06-22T16:35:51Z

Flaky test: #7592

…arch-project#7321) * Compress and cache cluster state during validate join request Signed-off-by: Aman Khare <[email protected]> * Add changelog and license Signed-off-by: Aman Khare <[email protected]> * Add javadoc and correct styling Signed-off-by: Aman Khare <[email protected]> * Add new handler for sending compressed cluster state in validate join flow and refactor code Signed-off-by: Aman Khare <[email protected]> * Refactor util method Signed-off-by: Aman Khare <[email protected]> * optimize imports Signed-off-by: Aman Khare <[email protected]> * Use cluster state version based cache instead of time based cache Signed-off-by: Aman Khare <[email protected]> * style fix Signed-off-by: Aman Khare <[email protected]> * fix styling 2 Signed-off-by: Aman Khare <[email protected]> * Use concurrent hashmap instead of cache, add UT class for ClusterStateUtils Signed-off-by: Aman Khare <[email protected]> * style fix Signed-off-by: Aman Khare <[email protected]> * Use AtomicReference instead of ConcurrentHashMap Signed-off-by: Aman Khare <[email protected]> * Use method overloading to simplify the caller code Signed-off-by: Aman Khare <[email protected]> * Resolve conflicts Signed-off-by: Aman Khare <[email protected]> * Change code structure to separate the flow for JoinHelper and PublicationTransportHelper Signed-off-by: Aman Khare <[email protected]> * Remove unnecessary input.setVersion line Co-authored-by: Andrew Ross <[email protected]> Signed-off-by: Aman Khare <[email protected]> --------- Signed-off-by: Aman Khare <[email protected]> Signed-off-by: Aman Khare <[email protected]> Co-authored-by: Aman Khare <[email protected]> Co-authored-by: Andrew Ross <[email protected]> (cherry picked from commit b17c88c)

Signed-off-by: Andrew Ross <[email protected]>

github-actions · 2023-06-22T16:39:10Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/18186/
CommitID: 0dfe96e
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-06-22T17:21:15Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.remotestore.SegmentReplicationUsingRemoteStoreIT.testPressureServiceStats

URL: https://build.ci.opensearch.org/job/gradle-check/18187/
CommitID: daeaa52
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <[email protected]>

This change was backported to 2.x in #7945. Signed-off-by: Andrew Ross <[email protected]>

…request (opensearch-project#7321) (opensearch-project#7945) * Compress and cache cluster state during validate join request (opensearch-project#7321) * Compress and cache cluster state during validate join request Signed-off-by: Aman Khare <[email protected]> * Add changelog and license Signed-off-by: Aman Khare <[email protected]> * Add javadoc and correct styling Signed-off-by: Aman Khare <[email protected]> * Add new handler for sending compressed cluster state in validate join flow and refactor code Signed-off-by: Aman Khare <[email protected]> * Refactor util method Signed-off-by: Aman Khare <[email protected]> * optimize imports Signed-off-by: Aman Khare <[email protected]> * Use cluster state version based cache instead of time based cache Signed-off-by: Aman Khare <[email protected]> * style fix Signed-off-by: Aman Khare <[email protected]> * fix styling 2 Signed-off-by: Aman Khare <[email protected]> * Use concurrent hashmap instead of cache, add UT class for ClusterStateUtils Signed-off-by: Aman Khare <[email protected]> * style fix Signed-off-by: Aman Khare <[email protected]> * Use AtomicReference instead of ConcurrentHashMap Signed-off-by: Aman Khare <[email protected]> * Use method overloading to simplify the caller code Signed-off-by: Aman Khare <[email protected]> * Resolve conflicts Signed-off-by: Aman Khare <[email protected]> * Change code structure to separate the flow for JoinHelper and PublicationTransportHelper Signed-off-by: Aman Khare <[email protected]> * Remove unnecessary input.setVersion line Co-authored-by: Andrew Ross <[email protected]> Signed-off-by: Aman Khare <[email protected]> --------- Signed-off-by: Aman Khare <[email protected]> Signed-off-by: Aman Khare <[email protected]> Co-authored-by: Aman Khare <[email protected]> Co-authored-by: Andrew Ross <[email protected]> (cherry picked from commit b17c88c) * Change version guard to 2.9.0 Signed-off-by: Andrew Ross <[email protected]> --------- Signed-off-by: Andrew Ross <[email protected]> Co-authored-by: Aman Khare <[email protected]>

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Rishab Nahata <[email protected]>

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <[email protected]>

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Shivansh Arora <[email protected]>

andrross requested review from Bukhtawar, CEHENKLE, Rishikesh1159, VachaShah, anasalkouz, dblock, dbwiddis, dreamer-89, gbbafna, kartg, kotwanikunal, mch2, nknize, owaiskazi19, reta, ryanbogan, saratvemulapalli, setiah, shwetathareja and tlfeng as code owners June 6, 2023 22:59

andrross mentioned this pull request Jun 6, 2023

TokenManager Interface #7452

Merged

6 tasks

andrross force-pushed the backport/backport-7321-to-2.x branch from bcde254 to 91f9db9 Compare June 8, 2023 18:02

shwetathareja reviewed Jun 14, 2023

View reviewed changes

server/src/main/java/org/opensearch/cluster/coordination/JoinHelper.java Show resolved Hide resolved

andrross force-pushed the backport/backport-7321-to-2.x branch from 91f9db9 to c785e9f Compare June 15, 2023 21:52

andrross requested a review from sachinpkale as a code owner June 15, 2023 21:52

andrross force-pushed the backport/backport-7321-to-2.x branch from c785e9f to fb56099 Compare June 16, 2023 23:33

andrross added a commit to andrross/OpenSearch that referenced this pull request Jun 20, 2023

Update compressed cluster state version guard

0f16817

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <[email protected]>

andrross mentioned this pull request Jun 20, 2023

Update compressed cluster state version guard #8185

Merged

6 tasks

kotwanikunal approved these changes Jun 20, 2023

View reviewed changes

andrross force-pushed the backport/backport-7321-to-2.x branch from fb56099 to 0dfe96e Compare June 21, 2023 15:27

amkhar and others added 2 commits June 22, 2023 09:36

Change version guard to 2.9.0

daeaa52

Signed-off-by: Andrew Ross <[email protected]>

andrross force-pushed the backport/backport-7321-to-2.x branch from 0dfe96e to daeaa52 Compare June 22, 2023 16:37

andrross merged commit 4b3b30d into opensearch-project:2.x Jun 22, 2023

andrross added a commit to andrross/OpenSearch that referenced this pull request Jun 22, 2023

Update compressed cluster state version guard

3daddc6

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <[email protected]>

kotwanikunal pushed a commit that referenced this pull request Jun 22, 2023

Update compressed cluster state version guard (#8185)

4be8918

This change was backported to 2.x in #7945. Signed-off-by: Andrew Ross <[email protected]>

sudarshan-baliga pushed a commit to Gaurav614/OpenSearch that referenced this pull request Jun 29, 2023

Update compressed cluster state version guard (opensearch-project#8185)

621c88a

This change was backported to 2.x in opensearch-project#7945. Signed-off-by: Andrew Ross <[email protected]>

andrross deleted the backport/backport-7321-to-2.x branch February 8, 2024 19:47

[Backport 2.x] Compress and cache cluster state during validate join request (#7321) #7945

[Backport 2.x] Compress and cache cluster state during validate join request (#7321) #7945

Uh oh!

Conversation

andrross commented Jun 6, 2023

Uh oh!

github-actions bot commented Jun 6, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 6, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

codecov bot commented Jun 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Jun 8, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

kotwanikunal commented Jun 13, 2023

Uh oh!

Uh oh!

github-actions bot commented Jun 15, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 16, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 17, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 21, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 21, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

andrross commented Jun 22, 2023

Uh oh!

github-actions bot commented Jun 22, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

github-actions bot commented Jun 22, 2023

Gradle Check (Jenkins) Run Completed with:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented Jun 6, 2023 •

edited

Loading