Fix race condition in SearchOnlyReplicaIT (#18072)

I observed a failure in local testing due to this issue. The code
previously was assuming it would observe a yellow cluster after
restarting a node, but the node could restart and re-join the cluster
before the assertion causing a failure. The `onNodeStopped` callback
lets the test execute code before the node is restarted, so it should
fix this issue.

Signed-off-by: Andrew Ross <[email protected]>

Author: andrross

server/src/internalClusterTest/java/org/opensearch/indices/settings/SearchOnlyReplicaIT.java

@@ -152,10 +152,15 @@ public void testFailoverWithSearchReplicaWhenSearchNodeRestarts() throws Excepti
         String searchNode = internalCluster().startSearchOnlyNode();
         ensureGreen(TEST_INDEX);
         // Restart Search Node
-        internalCluster().restartNode(searchNode);
-        // Ensure search shard is unassigned
-        ensureYellowAndNoInitializingShards(TEST_INDEX);
-        assertActiveSearchShards(0);
+        internalCluster().restartNode(searchNode, new InternalTestCluster.RestartCallback() {
+            @Override
+            public Settings onNodeStopped(String nodeName) throws Exception {
+                // Ensure search shard is unassigned
+                ensureYellowAndNoInitializingShards(TEST_INDEX);
+                assertActiveSearchShards(0);
+                return super.onNodeStopped(nodeName);
+            }
+        });
         // Ensure search shard is recovered
         ensureGreen(TEST_INDEX);
         assertActiveSearchShards(1);