Fix LeaderCheckerTests#testFollowerBehaviour (#83897) (#83952)

This test computes the expected message by tracking the different kinds
of failures generated by a mock transport service. This tracking counts
consecutive failures so is reset on success, but it is not explicitly
reset when starting the second pass. This was usually fine since we
start the second pass with a long sequence of checks that do not fail
which would reset the tracking. Rarely however this sequence was too
short to contain any success responses, so the counters would not be
reset as needed.

This commit makes sure we run at least `leaderCheckRetryCount`
iterations without a failure to ensure that at least one of them
succeeds.

Closes #83857
Closes #83924

Author: DaveCTurner

server/src/test/java/org/elasticsearch/cluster/coordination/LeaderCheckerTests.java

@@ -205,7 +205,9 @@ public String toString() {
         leaderChecker.updateLeader(leader2);
         {
             checkCount.set(0);
-            final long maxCheckCount = randomLongBetween(2, 1000);
+            // run at least leaderCheckRetryCount iterations to ensure at least one success so that we reset the counters and clear out
+            // anything left over from the previous run
+            final long maxCheckCount = randomLongBetween(leaderCheckRetryCount, 1000);
             logger.info("--> checking again that no failure is detected in {} checks", maxCheckCount);
             while (checkCount.get() < maxCheckCount) {
                 deterministicTaskQueue.runAllRunnableTasks();