Fix field_caps returning empty results for disable_objects mappings - #20814
Conversation
PR Reviewer Guide 🔍(Review updated until commit e212d0d)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to e212d0d Explore these optional code suggestions:
Previous suggestionsSuggestions up to commit 22919db
Suggestions up to commit b4eb2f0
Suggestions up to commit 0042299
Suggestions up to commit bd5b6aa
Suggestions up to commit 39e8551
|
|
❌ Gradle check result for a16b4cf: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
gaobinlong
left a comment
There was a problem hiding this comment.
The DCO check failed, please amend the commit message with your sign-off, add also add some change log for this PR. In addition, this bug fix is better to have some e2e yaml rest tests, see https://github.com/opensearch-project/OpenSearch/blob/main/TESTING.md#testing-the-rest-layer, could you add it, thanks! @newtonne
|
Thanks @gaobinlong. Sure, I can take a look at that. Would the rest tests be in addition to the current tests? |
Yeah, It's better to have yaml rest tests for API changes. |
a16b4cf to
39e8551
Compare
|
Persistent review updated to latest commit 39e8551 |
|
❌ Gradle check result for 39e8551: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
9dace53 to
bd5b6aa
Compare
|
Persistent review updated to latest commit 9dace53 |
|
Persistent review updated to latest commit bd5b6aa |
|
❌ Gradle check result for bd5b6aa: null Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
bd5b6aa to
0042299
Compare
|
Persistent review updated to latest commit 0042299 |
|
❌ Gradle check result for 0042299: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
0042299 to
b4eb2f0
Compare
|
Persistent review updated to latest commit b4eb2f0 |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #20814 +/- ##
============================================
- Coverage 73.27% 73.17% -0.10%
+ Complexity 72546 72431 -115
============================================
Files 5819 5819
Lines 331357 331360 +3
Branches 47877 47878 +1
============================================
- Hits 242796 242475 -321
- Misses 69077 69362 +285
- Partials 19484 19523 +39 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
b4eb2f0 to
22919db
Compare
|
Persistent review updated to latest commit 22919db |
|
❌ Gradle check result for 22919db: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
`_field_caps` returns an empty response for indexes where a
`disable_objects: true` object field has been populated with a child
field. A second bug causes such child fields to be corrupted after any
subsequent document is indexed into the same index - for example
`attributes.foo.bar` becomes `attributes.foo.foo.bar`.
## Root causes
When `_field_caps` walks the parent chain of a flattened leaf field
(e.g. `attributes.foo.bar`), it looks up an `ObjectMapper` for each
intermediate path. Under `disable_objects: true`, intermediate paths
like `attributes.foo` have no `ObjectMapper` by design - the fix adds a
null check to skip them and continue up the chain.
The field name corruption is caused by `ParametrizedFieldMapper.merge()`
using `name().lastIndexOf('.')` to reconstruct the parent `ContentPath`
when rebuilding a mapper. For a field with `simpleName` `foo.bar` and
full name `attributes.foo.bar`, this returns the position of the dot
before `bar` rather than the dot before `foo.bar`, so the parent path is
computed as `attributes.foo` instead of `attributes`. The fix computes
the boundary from `simpleName.length()` instead.
Signed-off-by: Cyrus Saeid <cyrus.s.dev@gmail.com>
22919db to
e212d0d
Compare
|
Persistent review updated to latest commit e212d0d |
…pensearch-project#20814) `_field_caps` returns an empty response for indexes where a `disable_objects: true` object field has been populated with a child field. A second bug causes such child fields to be corrupted after any subsequent document is indexed into the same index - for example `attributes.foo.bar` becomes `attributes.foo.foo.bar`. ## Root causes When `_field_caps` walks the parent chain of a flattened leaf field (e.g. `attributes.foo.bar`), it looks up an `ObjectMapper` for each intermediate path. Under `disable_objects: true`, intermediate paths like `attributes.foo` have no `ObjectMapper` by design - the fix adds a null check to skip them and continue up the chain. The field name corruption is caused by `ParametrizedFieldMapper.merge()` using `name().lastIndexOf('.')` to reconstruct the parent `ContentPath` when rebuilding a mapper. For a field with `simpleName` `foo.bar` and full name `attributes.foo.bar`, this returns the position of the dot before `bar` rather than the dot before `foo.bar`, so the parent path is computed as `attributes.foo` instead of `attributes`. The fix computes the boundary from `simpleName.length()` instead. Signed-off-by: Cyrus Saeid <cyrus.s.dev@gmail.com> Co-authored-by: Cyrus Saeid <cyrus.s.dev@gmail.com> Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com>
…pensearch-project#20814) `_field_caps` returns an empty response for indexes where a `disable_objects: true` object field has been populated with a child field. A second bug causes such child fields to be corrupted after any subsequent document is indexed into the same index - for example `attributes.foo.bar` becomes `attributes.foo.foo.bar`. ## Root causes When `_field_caps` walks the parent chain of a flattened leaf field (e.g. `attributes.foo.bar`), it looks up an `ObjectMapper` for each intermediate path. Under `disable_objects: true`, intermediate paths like `attributes.foo` have no `ObjectMapper` by design - the fix adds a null check to skip them and continue up the chain. The field name corruption is caused by `ParametrizedFieldMapper.merge()` using `name().lastIndexOf('.')` to reconstruct the parent `ContentPath` when rebuilding a mapper. For a field with `simpleName` `foo.bar` and full name `attributes.foo.bar`, this returns the position of the dot before `bar` rather than the dot before `foo.bar`, so the parent path is computed as `attributes.foo` instead of `attributes`. The fix computes the boundary from `simpleName.length()` instead. Signed-off-by: Cyrus Saeid <cyrus.s.dev@gmail.com> Co-authored-by: Cyrus Saeid <cyrus.s.dev@gmail.com>
Description
_field_capsreturns an empty response for indexes where adisable_objects: trueobject field has been populated with a child field. A second bug causes such child fields to be corrupted after any subsequent document is indexed into the same index - for exampleattributes.foo.barbecomesattributes.foo.foo.bar.Root causes
When
_field_capswalks the parent chain of a flattened leaf field (e.g.attributes.foo.bar), it looks up anObjectMapperfor each intermediate path. Underdisable_objects: true, intermediate paths likeattributes.foohave noObjectMapperby design - the fix adds a null check to skip them and continue up the chain.The field name corruption is caused by
ParametrizedFieldMapper.merge()usingname().lastIndexOf('.')to reconstruct the parentContentPathwhen rebuilding a mapper. For a field withsimpleNamefoo.barand full nameattributes.foo.bar, this returns the position of the dot beforebarrather than the dot beforefoo.bar, so the parent path is computed asattributes.fooinstead ofattributes. The fix computes the boundary fromsimpleName.length()instead.Related Issues
Resolves #20811
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.