Skip to content

Conversation

@yuancu
Copy link
Collaborator

@yuancu yuancu commented Oct 28, 2025

Description

This PR fixes a bug where nested fields of generated structs (like those created by the geoip() function) were not accessible, returning incorrect field names and null values.

Problem

When accessing nested fields from generated structs using dot notation, the system would:

  1. Return incorrect field names
  2. Return null values instead of the actual nested field data
  3. Return incorrect types

Solution

The fix involves two main changes:

QualifiedNameResolver.java:

  • Fixed the joinParts calculation logic when creating item access for nested fields
  • Added proper aliasing to preserve the original qualified field name in query results

Testing

  • Added integration tests in CalciteGeoIpFunctionsIT.java to verify the fix
  • Added YAML test cases to validate both logical and physical query plans
  • Tests confirm that accessing info.dummy_sub_field now returns the correct field name and value

Related Issues

Resolves #4682

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

LantaoJin
LantaoJin previously approved these changes Oct 28, 2025
@LantaoJin
Copy link
Member

LantaoJin commented Nov 3, 2025

Did your merge upstream action happen before #4716 being merged?

@qianheng-aws qianheng-aws merged commit 3a42c51 into opensearch-project:main Nov 3, 2025
31 of 33 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.19-dev failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.19-dev 2.19-dev
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.19-dev
# Create a new branch
git switch --create backport/backport-4683-to-2.19-dev
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 3a42c511cd08b2639f3c95441a9b7cac2f8ddb2c
# Push it to GitHub
git push --set-upstream origin backport/backport-4683-to-2.19-dev
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.19-dev

Then, create a pull request where the base branch is 2.19-dev and the compare/head branch is backport/backport-4683-to-2.19-dev.

yuancu added a commit to yuancu/sql-plugin that referenced this pull request Nov 3, 2025
* Correct subfield access logical when calling ITEM

Signed-off-by: Yuanchun Shen <[email protected]>

* Add explain and integration tests

Signed-off-by: Yuanchun Shen <[email protected]>

---------

Signed-off-by: Yuanchun Shen <[email protected]>
(cherry picked from commit 3a42c51)
@yuancu yuancu deleted the issues/4682 branch November 4, 2025 02:14
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.19-dev failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.19-dev 2.19-dev
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.19-dev
# Create a new branch
git switch --create backport/backport-4683-to-2.19-dev
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 3a42c511cd08b2639f3c95441a9b7cac2f8ddb2c
# Push it to GitHub
git push --set-upstream origin backport/backport-4683-to-2.19-dev
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.19-dev

Then, create a pull request where the base branch is 2.19-dev and the compare/head branch is backport/backport-4683-to-2.19-dev.

yuancu added a commit to yuancu/sql-plugin that referenced this pull request Nov 4, 2025
* Correct subfield access logical when calling ITEM

Signed-off-by: Yuanchun Shen <[email protected]>

* Add explain and integration tests

Signed-off-by: Yuanchun Shen <[email protected]>

---------

Signed-off-by: Yuanchun Shen <[email protected]>
(cherry picked from commit 3a42c51)
yuancu added a commit to yuancu/sql-plugin that referenced this pull request Nov 4, 2025
* Correct subfield access logical when calling ITEM

Signed-off-by: Yuanchun Shen <[email protected]>

* Add explain and integration tests

Signed-off-by: Yuanchun Shen <[email protected]>

---------

Signed-off-by: Yuanchun Shen <[email protected]>
(cherry picked from commit 3a42c51)
LantaoJin pushed a commit that referenced this pull request Nov 4, 2025
) (#4721)

* Fix sub-fields accessing of generated structs (#4683)

* Correct subfield access logical when calling ITEM

Signed-off-by: Yuanchun Shen <[email protected]>

* Add explain and integration tests

Signed-off-by: Yuanchun Shen <[email protected]>

---------

Signed-off-by: Yuanchun Shen <[email protected]>
(cherry picked from commit 3a42c51)

* Mask host position in the plan of testInternalItemAccessOnStructs

Signed-off-by: Yuanchun Shen <[email protected]>

---------

Signed-off-by: Yuanchun Shen <[email protected]>
@LantaoJin LantaoJin added the backport-manually Filed a PR to backport manually. label Nov 4, 2025
expani pushed a commit to vinaykpud/sql that referenced this pull request Nov 4, 2025
* Correct subfield access logical when calling ITEM

Signed-off-by: Yuanchun Shen <[email protected]>

* Add explain and integration tests

Signed-off-by: Yuanchun Shen <[email protected]>

---------

Signed-off-by: Yuanchun Shen <[email protected]>
sandeshkr419 added a commit to sandeshkr419/sql that referenced this pull request Dec 3, 2025
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Simeon Widdis <[email protected]>
Co-authored-by: Manasvini B S <[email protected]>
Co-authored-by: opensearch-ci-bot <[email protected]>
Co-authored-by: Louis Chu <[email protected]>
Co-authored-by: Chen Dai <[email protected]>
Co-authored-by: Mebsina <[email protected]>
Co-authored-by: Yuanchun Shen <[email protected]>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: Kai Huang <[email protected]>
Co-authored-by: Peng Huo <[email protected]>
Co-authored-by: Alexey Temnikov <[email protected]>
Co-authored-by: Riley Jerger <[email protected]>
Co-authored-by: Tomoyuki MORITA <[email protected]>
Co-authored-by: Lantao Jin <[email protected]>
Co-authored-by: Songkan Tang <[email protected]>
Co-authored-by: qianheng <[email protected]>
Co-authored-by: Simeon Widdis <[email protected]>
Co-authored-by: Xinyuan Lu <[email protected]>
Co-authored-by: Jialiang Liang <[email protected]>
Co-authored-by: Peter Zhu <[email protected]>
Co-authored-by: Vinay Krishna Pudyodu <[email protected]>
Co-authored-by: expani <[email protected]>
Co-authored-by: expani1729 <[email protected]>
Co-authored-by: Vamsi Manohar <[email protected]>
Co-authored-by: ritvibhatt <[email protected]>
Co-authored-by: Xinyu Hao <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Co-authored-by: Marc Handalian <[email protected]>
Fix join type ambiguous issue when specify the join type with sql-like join criteria (opensearch-project#4474)
Fix issue 4441 (opensearch-project#4449)
Fix missing keywordsCanBeId (opensearch-project#4491)
Fix the bug of explicit makeNullLiteral for UDT fields (opensearch-project#4475)
Fix mapping after aggregation push down (opensearch-project#4500)
Fix percentile bug (opensearch-project#4539)
Fix JsonExtractAllFunctionIT failure (opensearch-project#4556)
Fix sort push down into agg after project already pushed (opensearch-project#4546)
Fix push down failure for min/max on derived field (opensearch-project#4572)
Fix compile issue in main (opensearch-project#4608)
Fix filter parsing failure on date fields with non-default format (opensearch-project#4616)
Fix bin nested fields issue (opensearch-project#4606)
Fix: Support Alias Fields in MIN, MAX, FIRST, LAST, and TAKE Aggregations (opensearch-project#4621)
fix rename issue (opensearch-project#4670)
Fixes for `Multisearch` and `Append` command (opensearch-project#4512)
Fix asc/desc keyword behavior for sort command (opensearch-project#4651)
Fix] Fix unexpected shift of extraction for `rex` with nested capture groups in named groups  (opensearch-project#4641)
Fix CVE-2025-48924 (opensearch-project#4665)
Fix sub-fields accessing of generated structs (opensearch-project#4683)
Fix] Incorrect Field Index Mapping in AVG to SUM/COUNT Conversion (opensearch-project#15)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport 2.19-dev backport-failed backport-manually Filed a PR to backport manually. bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Nested fields of generated structs not accessible

3 participants