[CI] IndexResolutionIT testDoesNotResolveClosedIndex failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-periodic #10035 / openjdk21_checkpart3_java-fips-matrix
• elasticsearch-intake #26943 / part3

Reproduction Line:

./gradlew ":x-pack:plugin:esql:internalClusterTest" --tests "org.elasticsearch.xpack.esql.plugin.IndexResolutionIT.testDoesNotResolveClosedIndex" -Dtests.seed=5E488968D1D289DA -Dtests.locale=fr-TN -Dtests.timezone=Etc/GMT+5 -Druntime.java=21 -Dtests.fips.enabled=true

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: CloseIndexResponse failed - not acked
Expected: <true>
     but: was <false>

Issue Reasons:

• [main] 2 failures in test testDoesNotResolveClosedIndex (0.9% fail rate in 223 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 2 failures in test testDoesNotResolveClosedIndex (0.9% fail rate in 223 executions)

Build Scans:

• elasticsearch-periodic #10035 / openjdk21_checkpart3_java-fips-matrix
• elasticsearch-intake #26943 / part3

[elasticsearchmachine]

Pinging @elastic/es-analytical-engine (Team:Analytics)

[idegtiarenko]

The log says:

[2025-08-15T01:49:17,955][INFO ][o.e.c.m.MetadataIndexStateService][node_s0][masterService#updateTask][T#1] closing indices [index-1/OdPRib6vT5ipq7_-t2xVgg]	
[2025-08-15T01:49:18,193][INFO ][o.e.c.m.MetadataIndexStateService][node_s0][masterService#updateTask][T#1] completed closing of indices []

It is not clear why the index was not closed.
Since I was not able to reproduce it locally, I am going to un-mute the test and enable additional logging.

[idegtiarenko]

There is a first failure with additional logging:

[2025-08-21T09:06:53,781][INFO ][o.e.c.m.MetadataIndexStateService][node_s0][masterService#updateTask][T#1] closing indices [index-1/3TbT8mg2SV2MKrh76uE5dw]	
[2025-08-21T09:06:53,825][DEBUG][o.e.c.m.MetadataIndexStateService][node_s0][masterService#updateTask][T#1] verification of shards before closing [index-1/3TbT8mg2SV2MKrh76uE5dw] failed [[index-1/3TbT8mg2SV2MKrh76uE5dw]={"index-1":{"closed":false,"failedShards":{"6":{"failures":[{"node":"JIBP7kHPSUGUj5jwEpGypw","shard":6,"index":"index-1","status":"INTERNAL_SERVER_ERROR","reason":{"type":"illegal_state_exception","reason":"Global checkpoint [-1] mismatches maximum sequence number [0] on index shard [index-1][6]"}}]}}}}]	
[2025-08-21T09:06:53,825][INFO ][o.e.c.m.MetadataIndexStateService][node_s0][masterService#updateTask][T#1] completed closing of indices []