Skip to content

Comments

[ES|QL] Do not skip a remote cluster base on the query's execution time status#138332

Merged
fang-xing-esql merged 15 commits intoelastic:mainfrom
fang-xing-esql:snapshot-cluster-status-before-execution
Dec 4, 2025
Merged

[ES|QL] Do not skip a remote cluster base on the query's execution time status#138332
fang-xing-esql merged 15 commits intoelastic:mainfrom
fang-xing-esql:snapshot-cluster-status-before-execution

Conversation

@fang-xing-esql
Copy link
Member

@fang-xing-esql fang-xing-esql commented Nov 20, 2025

This PR attempts to prevent a fork or subquery branch from being skipped on a remote cluster when the query's execution status is changed(to SUCCESSFUL or PARTIAL for example) at execution time, and makes sure the results are collected from all branches executed in parallel. It does not correct the execution status or metrics collected on fork or subquery.

This PR is a prerequisite for enabling CCS tests for fork and subquery (#137776). Without this change, the results of these CCS tests can be non-deterministic because some branches may be skipped unexpectedly, and cause incorrect query results. This fix can either be merged together with #137776 or submitted by itself. This change is behind snapshot and is applied to branches from fork and subquery only, so that the CCS tests for fork and subquery can return consistent and correct results. The existing CCS tests for queries without fork or subquery branches should pass.

We might need new data structure to support collecting correct execution status and metrics for fork and subquery, and it will be addressed in a separate PR.

@elasticsearchmachine
Copy link
Collaborator

Hi @fang-xing-esql, I've created a changelog YAML for you.

@fang-xing-esql fang-xing-esql added the test-release Trigger CI checks against release build label Nov 25, 2025
@fang-xing-esql fang-xing-esql marked this pull request as ready for review November 25, 2025 14:52
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Nov 25, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@fang-xing-esql
Copy link
Member Author

The two failed release tests happen on main as well, they are not introduced by this PR.

Copy link
Contributor

@ioanatia ioanatia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! is the next step to enable the multi cluster tests running with snapshot builds for FORK and views?

@fang-xing-esql
Copy link
Member Author

looks good! is the next step to enable the multi cluster tests running with snapshot builds for FORK and views?

Thank you for reviewing @ioanatia ! Yes, the next step is to enable MultiClusterSpecIT for fork, subquery and view.

@fang-xing-esql fang-xing-esql removed the test-release Trigger CI checks against release build label Dec 3, 2025
@fang-xing-esql fang-xing-esql merged commit 21d031b into elastic:main Dec 4, 2025
34 checks passed
@fang-xing-esql fang-xing-esql deleted the snapshot-cluster-status-before-execution branch January 30, 2026 14:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants