Skip to content

[Segment Replication]. Implement peer copy as a source of replication. #3322

@mch2

Description

@mch2

Segrep needs a service and handlers similar to PeerRecoveryTargetService to manage copying of segments between replicas and primaries.

The service should first fetch the latest checkpoint metadata from a primary shard and then kick off sending of segments back to the replica.

This is similar to the PrimaryShardReplicationSource code written as part of the POC. This POC code has a lot of duplication with PeerRecoveryTargetService, particularly in the endpoint to copy file chunks.

We will also need a service similar to PeerRecoverySourceService that handles requests from replicas. With Segment replication this source service would handle requests for new checkpoints (GET_CHECKPOINT in poc) and a handler for a request for files to be copied (GET_FILES in poc). Once a checkpoint request is received we need to incref the requested segments to ensure they are not merged away during a copy. The GET_FILES endpoint would start a copy process back to replicas, sending file chunks.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions