[9.3] ESQL: Fix incorrectly optimized fork with nullify unmapped_fields (#143030)#144205
Closed
kanoshiou wants to merge 1 commit intoelastic:9.3from
Closed
[9.3] ESQL: Fix incorrectly optimized fork with nullify unmapped_fields (#143030)#144205kanoshiou wants to merge 1 commit intoelastic:9.3from
kanoshiou wants to merge 1 commit intoelastic:9.3from
Conversation
…astic#143030) This PR fixes a bug where `Fork.withSubPlans()` incorrectly reassigned new `NameId`s to its output attributes, breaking references in the upper plan. This issue specifically manifests when using `FORK` alongside the `SET unmapped_fields="nullify"` mode. By design, a `FORK` assigns new `NameId`s to its output attributes via `refreshOutput()` to decouple them from the internal branches. This isolation is necessary to prevent unintended side effects during plan optimizations, such as aggressive constant folding leaking across branches. However, the previous implementation unconditionally re-minted these `NameId`s every time `withSubPlans()` was called. Because of this, any node sitting above the `FORK` (like `EVAL` or `STATS`) that already held a reference to the initial `NameId`s would suddenly point to a nonexistent ID. Downstream analysis rules would then fail to resolve these orphaned references, causing the plan execution to fail with an *"optimized incorrectly due to missing references"* error. Fixes elastic#142762 (cherry picked from commit 5fb7136) # Conflicts: # muted-tests.yml # x-pack/plugin/esql/qa/server/src/main/java/org/elasticsearch/xpack/esql/qa/rest/generative/GenerativeRestTest.java # x-pack/plugin/esql/qa/testFixtures/src/main/resources/unmapped-nullify.csv-spec # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/action/EsqlCapabilities.java # x-pack/plugin/esql/src/main/java/org/elasticsearch/xpack/esql/analysis/rules/ResolveUnmapped.java # x-pack/plugin/esql/src/test/java/org/elasticsearch/xpack/esql/analysis/AnalyzerUnmappedTests.java
elasticsearchmachine
added
external-contributor
astefan
added
>bug
:Analytics/ES|QL
Contributor
|
buildkite test this |
Contributor
Author
|
My apologies,@astefan. I am currently in the hospital and trying to manage this cherry-pick remotely from my phone is proving too difficult. My local tests are failing, and I’m unable to properly track this backport under these conditions. I'm sorry for can't be able to backport this patch this as promised.😢 |
Contributor
|
No worries @kanoshiou. Thank you for the update. I'll take a look myself. Take care! |
Contributor
|
Actually, the backport would be slightly easier if we wait for #144210 to be merged as well |
Contributor
|
Update: contrary to what I previously said on the original PR, we had to change plans and urgently backport #143399 after all. I'm sorry for the noise, @kanoshiou and @astefan !
Oh no, get well soon, @kanoshiou ! |
Contributor
|
Backported with #144386 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto9.3:Questions ?
Please refer to the Backport tool documentation