HDDS-7930. input stream does not refresh expired block token. #4378
Conversation
Change-Id: I6da102acc0a8ab2873388bf3032d373de5945bf5
adoroszlai
left a comment
adoroszlai
left a comment
There was a problem hiding this comment.
LGTM, with minor change suggested for logging.
| LOG.debug("New block location info for block {}: {}", | ||
| blockID, blockLocationInfo); |
There was a problem hiding this comment.
BlockLocationInfo#toString includes the token. I think we should keep logging the pipeline only.
There was a problem hiding this comment.
omg i missed this one. Will add an addendum to it.
szetszwo
left a comment
szetszwo
left a comment
There was a problem hiding this comment.
+1 the change looks good.
|
Refresh pipeline is used on failover, once the improved block token design is in the performance of refresh of block tokens won't be an issue. I guess for the short time this is ok to refresh block tokens along with pipeline. |
The refresh function calls getKeyInfo and OM generates a new block token every refresh already. Before this change, the client input stream doesn't take the new block token in the result. (And that sounds like a bug to me) |
|
Merged. Thanks all @szetszwo @duongkame @kerneltime @adoroszlai |
* master: (262 commits) HDDS-8153. Integrate ContainerBalancer with MoveManager (apache#4391) HDDS-8090. When getBlock from a datanode fails, retry other datanodes. (apache#4357) HDDS-8163 Use try-with-resources to ensure close rockdb connection in SstFilteringService (apache#4402) HDDS-8065. Provide GNU long options (apache#4394) HDDS-7930. [addendum] input stream does not refresh expired block token. HDDS-7930. input stream does not refresh expired block token. (apache#4378) HDDS-7740. [Snapshot] Implement SnapshotDeletingService (apache#4244) HDDS-8076. Use container cache in Key listing API. (apache#4346) HDDS-8091. [addendum] Generate list of config tags from ConfigTag enum - Hadoop 3.1 compatibility fix (apache#4374) HDDS-8144. TestDefaultCertificateClient#testTimeBeforeExpiryGracePeriod fails as we approach DST. (apache#4382) HDDS-8151. Support fine grained lifetime for root CA certificate (apache#4386) HDDS-8150. RpcClientTest and ConfigurationSourceTest not run due to naming convention (apache#4388) HDDS-8131. Add Configuration for OM Ratis Log Purge Tuning Parameters. (apache#4371) HDDS-8133. Create ozone sh key checksum command (apache#4375) HDDS-8142. Check if no entries in Block DB for a container on container delete (apache#4379) HDDS-8118. Fail container delete on non empty chunks dir (apache#4367) HDDS-8028. JNI for RocksDB SST Dump tool (apache#4315) HDDS-8129. ContainerStateMachine allows two different tasks with the same container id running in parallel. (apache#4370) HDDS-8119. Remove loosely related AutoCloseable from SendContainerOutputStream (apache#4368) close db connection (apache#4366) ...
5 participants
What changes were proposed in this pull request?
Update the retry function to not only update pipeline, but also updating block token.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7930
How was this patch tested?
Unit tests