HDDS-4176. Fix failed UT: test2WayCommitForTimeoutException

[runzhiwang]

What changes were proposed in this pull request?

What's the problem ?

What's the reason of the failed ut ?

When cluster create 9 nodes, the following code try to shutdown the follower of the pipeline. Actually, the pipeline only exist in 3 nodes, but the following code check 9 nodes, so the pipeline can not found in the other 6 nodes.

    for (HddsDatanodeService dn : cluster.getHddsDatanodes()) {
      // shutdown the ratis follower
      if (ContainerTestHelper.isRatisFollower(dn, pipeline)) {
        cluster.shutdownHddsDatanode(dn.getDatanodeDetails());
        break;
      }
    }

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-4176

How was this patch tested?

Existed ut.

[runzhiwang]

@elek Could you help review this patch ? Thank you very much.

[amaliujia]

This logic looks right to me.

Can you bump up the number of DN to make tests run against higher number DN: https://github.com/apache/hadoop-ozone/blob/34ee8311b0d0a37878fe1fd2e5d8c1b91aa8cc8f/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/Test2WayCommitInRatis.java#L109

Which seems will cover your change?

[runzhiwang]

@amaliujia Thanks for your review. TestWatchForCommit already startCluster with 9 nodes, maybe it is enough.
https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java#L124

[amaliujia]

Oops. You are right. I was looking at a wrong file.

[adoroszlai]

Thanks @runzhiwang for fixing this long standing intermittent failure.

[adoroszlai]

Nit: nodesInPipeline may be a better name.

[runzhiwang]

@adoroszlai Thanks for review. I have updated the patch.

[adoroszlai]

Thanks @runzhiwang for the fix and @amaliujia for the review.