Remote replication and recovery changes#20424

Merged

bharath-techie merged 26 commits intoopensearch-project:feature/datafusionfrom

raghuvanshraj:feature/remote-replication

Jan 20, 2026

Commits on Jan 19, 2026

Changes to get remote upload and replication to work for lucene indices
raghuvanshraj
committed
Replication changes
raghuvanshraj
committed
Fixes for remote store integration (#20325)

mgodwan
authored and
raghuvanshraj
committed
Updated and Refactored the code to have only required changes for recovery

Kamal Nayan
authored and
raghuvanshraj
committed
Removed extra logging lines

Kamal Nayan
authored and
raghuvanshraj
committed
Resolving PR comments

Kamal Nayan
authored and
raghuvanshraj
committed
Updated the VSRManagerTests to use ParquetFileMetadata as flushResult

Kamal Nayan
authored and
raghuvanshraj
committed
Fixing RemoteSegmentStoreDirectoryTests and RemoteSegmentStoreDirectoryWithPinnedTimestampTests

Kamal Nayan
authored and
raghuvanshraj
committed
Fix for merged files not getting uploaded/deleted
raghuvanshraj
committed
Added support for SSE KMS testing via run.gradle
raghuvanshraj
committed
[RemoteStore] Add support for repository with server side encryption enabled to avoid client side encryption (#19630)

pranikum
authored and
raghuvanshraj
committed
Adding support for SSE KMS in RemoteSegmentStoreDirectoryFactory
raghuvanshraj
committed
Fixing lucene remote recovery and files should not get written in replicas

Kamal Nayan
authored and
raghuvanshraj
committed
Removing extra logging statements

Kamal Nayan
authored and
raghuvanshraj
committed
Revert "Fixing lucene remote recovery and files should not get written in replicas"
raghuvanshraj
committed
Using NIOFSDirectory in GenericStoreDirectory
raghuvanshraj
committed
Adding NRTReplicationCompositeEngine for checkpoint tracking during recovery
raghuvanshraj
committed
Using index input for checksum calculation instead of input stream
raghuvanshraj
committed
Fixing lucene remote recovery and files should not get written in replicas

Kamal Nayan
authored and
raghuvanshraj
committed
Fixed RemoteIndexRecoveryIT::testRerouteRecovery

Kamal Nayan
authored and
raghuvanshraj
committed
Removed the isReadonly check from compositeEngine as we have the NRTEngine now

Kamal Nayan
authored and
raghuvanshraj
committed
Removing override for getHistoryUUID in NRTReplicationCompositeEngine
raghuvanshraj
committed
Updated the IndexFileDeleterTests to use absolute file path
Kamal Nayan
committed
Refactoring syncSegmentsFromGivenRemoteSegmentStore and syncSegmentsFromRemoteSegmentStore APIs
Kamal Nayan
committed
Made deleteUnrefrencedFiles api more generic
Kamal Nayan
committed

Commits on Jan 20, 2026

Modifying UUID to be added before file name extension for catalog support on parquet files
raghuvanshraj
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Remote replication and recovery changes#20424

Remote replication and recovery changes#20424
bharath-techie merged 26 commits intoopensearch-project:feature/datafusionfrom
raghuvanshraj:feature/remote-replication

Commits on Jan 19, 2026

Changes to get remote upload and replication to work for lucene indices

Replication changes

Fixes for remote store integration (#20325)

Updated and Refactored the code to have only required changes for recovery

Removed extra logging lines

Resolving PR comments

Updated the VSRManagerTests to use ParquetFileMetadata as flushResult

Fixing RemoteSegmentStoreDirectoryTests and RemoteSegmentStoreDirectoryWithPinnedTimestampTests

Fix for merged files not getting uploaded/deleted

Added support for SSE KMS testing via run.gradle

[RemoteStore] Add support for repository with server side encryption enabled to avoid client side encryption (#19630)

Adding support for SSE KMS in RemoteSegmentStoreDirectoryFactory

Fixing lucene remote recovery and files should not get written in replicas

Removing extra logging statements

Revert "Fixing lucene remote recovery and files should not get written in replicas"

Using NIOFSDirectory in GenericStoreDirectory

Adding NRTReplicationCompositeEngine for checkpoint tracking during recovery

Using index input for checksum calculation instead of input stream

Fixing lucene remote recovery and files should not get written in replicas

Fixed RemoteIndexRecoveryIT::testRerouteRecovery

Removed the isReadonly check from compositeEngine as we have the NRTEngine now

Removing override for getHistoryUUID in NRTReplicationCompositeEngine

Updated the IndexFileDeleterTests to use absolute file path

Refactoring syncSegmentsFromGivenRemoteSegmentStore and syncSegmentsFromRemoteSegmentStore APIs

Made deleteUnrefrencedFiles api more generic

Commits on Jan 20, 2026

Modifying UUID to be added before file name extension for catalog support on parquet files