Skip to content

Conversation

@kaijchen
Copy link
Member

@kaijchen kaijchen commented Apr 26, 2022

What changes were proposed in this pull request?

When replicating EC containers, we need more rpc than copy container.

CoordinatorDN -> HealthyDN

This patch adds support for ContainerCommand in replication service.

Server Side:

  1. Added command handler to dispatch container command requests.

Client Side:

  1. Added readContainer() to get number of total blocks to list.
  2. Added listBlock() to list blocks in a container.
  3. Added readChunk() to read the data from healthy datanodes.

CoordinatorDN -> TargetDN

The CoordinatorDN should push the chunks to the TargetDN after computing missing blocks.
WriteChunkProto without data means the end of the container.

Push is chosen for following reasons:

  1. CoordinatorDN does not need to hold the missing blocks for long.
  2. CoordinatorDN can controll the progress of the recovery.

Server Side:

  1. Added dummy command handler for push chunk and start recovery.

Client Side:

  1. Added writeChunk() to push chunks to target datanodes.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-6518

How was this patch tested?

New integration test: TestReplicationService

@kaijchen kaijchen marked this pull request as draft April 26, 2022 08:48
@kaijchen kaijchen changed the title HDDS-6518. EC: Support ContainerCommand in intra-datanode grpc service HDDS-6518. EC: Support ContainerCommand in replication service Apr 26, 2022
@kaijchen kaijchen marked this pull request as ready for review April 26, 2022 09:23
@kaijchen
Copy link
Member Author

Cc @umamaheswararao @guihecheng for review.

@kaijchen kaijchen changed the title HDDS-6518. EC: Support ContainerCommand in replication service HDDS-6518. EC: Add rpc for EC recovery in replication service Apr 27, 2022
@kaijchen
Copy link
Member Author

Closed because we decided to reuse client logic for offline recovery.

@kaijchen kaijchen closed this May 11, 2022
@kaijchen kaijchen deleted the HDDS-6518 branch August 2, 2022 03:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant