Remote replication and recovery changes by raghuvanshraj · Pull Request #20424 · opensearch-project/OpenSearch

raghuvanshraj · 2026-01-15T06:09:19Z

Description

~~Known bug in this PR: Checkpoint tracking during remote recovery is buggy, leading to segments getting recreated from translog.~~

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

coderabbitai · 2026-01-15T06:09:25Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

github-actions · 2026-01-15T06:14:41Z

❌ Gradle check result for 1f3aa80: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-01-15T07:32:04Z

❌ Gradle check result for 31041da: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-01-15T10:26:01Z

❌ Gradle check result for 839a560: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-01-16T12:43:58Z

❌ Gradle check result for b576be5: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-01-16T13:06:28Z

❌ Gradle check result for d00f2b9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-01-16T13:34:11Z

❌ Gradle check result for f62dce8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

…ryWithPinnedTimestampTests

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

…enabled to avoid client side encryption (opensearch-project#19630) Signed-off-by: Pranit Kumar <pranikum@amazon.com>

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

…licas

…n in replicas" This reverts commit 1f3aa80.

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

…ecovery Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

…licas

…ngine now

github-actions · 2026-01-19T11:28:53Z

❌ Gradle check result for 760f2c3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

github-actions · 2026-01-19T12:15:56Z

❌ Gradle check result for 9926737: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-01-19T13:06:29Z

❌ Gradle check result for 817c024: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…romRemoteSegmentStore APIs

github-actions · 2026-01-19T13:35:20Z

❌ Gradle check result for 76fc740: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-01-19T14:05:02Z

❌ Gradle check result for dd5463e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

…port on parquet files Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

github-actions · 2026-01-20T10:36:17Z

❌ Gradle check result for 7341e0f: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

* Changes to get remote upload and replication to work for lucene indices Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> * Replication changes Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> * Fixes for remote store integration (opensearch-project#20325) Signed-off-by: Mohit Godwani <mgodwan@amazon.com> * Updated and Refactored the code to have only required changes for recovery * Removed extra logging lines * Resolving PR comments * Updated the VSRManagerTests to use ParquetFileMetadata as flushResult * Fixing RemoteSegmentStoreDirectoryTests and RemoteSegmentStoreDirectoryWithPinnedTimestampTests * Fix for merged files not getting uploaded/deleted Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> * Added support for SSE KMS testing via run.gradle Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> * [RemoteStore] Add support for repository with server side encryption enabled to avoid client side encryption (opensearch-project#19630) Signed-off-by: Pranit Kumar <pranikum@amazon.com> * Adding support for SSE KMS in RemoteSegmentStoreDirectoryFactory Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> * Fixing lucene remote recovery and files should not get written in replicas * Removing extra logging statements * Revert "Fixing lucene remote recovery and files should not get written in replicas" This reverts commit 1f3aa80. * Using NIOFSDirectory in GenericStoreDirectory Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> * Adding NRTReplicationCompositeEngine for checkpoint tracking during recovery Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> * Using index input for checksum calculation instead of input stream Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> * Fixing lucene remote recovery and files should not get written in replicas * Fixed RemoteIndexRecoveryIT::testRerouteRecovery * Removed the isReadonly check from compositeEngine as we have the NRTEngine now * Removing override for getHistoryUUID in NRTReplicationCompositeEngine Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> * Updated the IndexFileDeleterTests to use absolute file path * Refactoring syncSegmentsFromGivenRemoteSegmentStore and syncSegmentsFromRemoteSegmentStore APIs * Made deleteUnrefrencedFiles api more generic * Modifying UUID to be added before file name extension for catalog support on parquet files Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> --------- Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com> Signed-off-by: Mohit Godwani <mgodwan@amazon.com> Signed-off-by: Pranit Kumar <pranikum@amazon.com> Co-authored-by: Mohit Godwani <81609427+mgodwan@users.noreply.github.com> Co-authored-by: Kamal Nayan <askkamal@amazon.com> Co-authored-by: pranikum <kpranit81@gmail.com>

raghuvanshraj requested review from Bukhtawar, CEHENKLE, Rishikesh1159, anasalkouz, andrross, ashking94, cwperks, dbwiddis, gbbafna, jed326, kotwanikunal, mch2, msfroh, owaiskazi19, reta, sachinpkale and saratvemulapalli as code owners January 15, 2026 06:09

raghuvanshraj requested review from a team, shwetathareja and sohami as code owners January 15, 2026 06:09

raghuvanshraj force-pushed the feature/remote-replication branch from b576be5 to d00f2b9 Compare January 16, 2026 13:01

raghuvanshraj added 2 commits January 19, 2026 16:52

Changes to get remote upload and replication to work for lucene indices

c8812d1

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

Replication changes

c2e9231

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

Kamal Nayan and others added 16 commits January 19, 2026 16:52

Resolving PR comments

b3e46a5

Updated the VSRManagerTests to use ParquetFileMetadata as flushResult

c5eea06

Fixing RemoteSegmentStoreDirectoryTests and RemoteSegmentStoreDirecto…

7925ce3

…ryWithPinnedTimestampTests

Fix for merged files not getting uploaded/deleted

7a5c4b0

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

Added support for SSE KMS testing via run.gradle

799279f

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

[RemoteStore] Add support for repository with server side encryption …

5a56859

…enabled to avoid client side encryption (opensearch-project#19630) Signed-off-by: Pranit Kumar <pranikum@amazon.com>

Adding support for SSE KMS in RemoteSegmentStoreDirectoryFactory

3e24e1c

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

Fixing lucene remote recovery and files should not get written in rep…

db2b6c5

…licas

Removing extra logging statements

b444e6c

Revert "Fixing lucene remote recovery and files should not get writte…

c67d004

…n in replicas" This reverts commit 1f3aa80.

Using NIOFSDirectory in GenericStoreDirectory

9483cbb

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

Adding NRTReplicationCompositeEngine for checkpoint tracking during r…

93911e1

…ecovery Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

Using index input for checksum calculation instead of input stream

d638436

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

Fixing lucene remote recovery and files should not get written in rep…

caef264

…licas

Fixed RemoteIndexRecoveryIT::testRerouteRecovery

d7d5220

Removed the isReadonly check from compositeEngine as we have the NRTE…

760f2c3

…ngine now

raghuvanshraj force-pushed the feature/remote-replication branch from dda84c3 to 760f2c3 Compare January 19, 2026 11:22

Removing override for getHistoryUUID in NRTReplicationCompositeEngine

9926737

Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

Updated the IndexFileDeleterTests to use absolute file path

817c024

Refactoring syncSegmentsFromGivenRemoteSegmentStore and syncSegmentsF…

76fc740

…romRemoteSegmentStore APIs

Made deleteUnrefrencedFiles api more generic

dd5463e

Modifying UUID to be added before file name extension for catalog sup…

7341e0f

…port on parquet files Signed-off-by: Raghuvansh Raj <raghraaj@amazon.com>

bharath-techie merged commit b4ff131 into opensearch-project:feature/datafusion Jan 20, 2026
8 of 30 checks passed

Conversation

raghuvanshraj commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

coderabbitai bot commented Jan 15, 2026

Review skipped

Uh oh!

github-actions bot commented Jan 15, 2026

Uh oh!

github-actions bot commented Jan 15, 2026

Uh oh!

github-actions bot commented Jan 15, 2026

Uh oh!

github-actions bot commented Jan 16, 2026

Uh oh!

github-actions bot commented Jan 16, 2026

Uh oh!

github-actions bot commented Jan 16, 2026

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

github-actions bot commented Jan 19, 2026

Uh oh!

Uh oh!

github-actions bot commented Jan 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Comments

raghuvanshraj commented Jan 15, 2026 •

edited

Loading