Updating the FieldMappers to insert call addField of DocumentInput by darjisagar7 · Pull Request #21001 · opensearch-project/OpenSearch

darjisagar7 · 2026-03-26T05:30:32Z

Description

This PR adds the following functionalities:

Introducing a new Feature Flag for the Pluggable dataformat feature. [RFC] OpenSearch Engine: Pluggable Component Bundling #20644
It adds a new IndexSettings named "index.pluggable.dataformat.enabled", which can be used along with feature flag to enable the pluggable dataformat for an Index.
This PR also ensures that the FieldMapper are decoupled from the Lucene's code. Based on the pluggable dataformat architecture the data will be added to DocumentInput which will add the data to respective dataformat.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2026-03-26T05:31:33Z

Failed to generate code suggestions for PR

github-actions · 2026-03-26T06:47:58Z

❌ Gradle check result for 7f26163: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-03-26T07:46:05Z

PR Code Analyzer ❗

AI-powered 'Code-Diff-Analyzer' found issues on commit babb436.

Path	Line	Severity	Description
server/src/test/java/org/opensearch/index/mapper/FakeStringFieldMapper.java	134	low	parseCreateFieldForPluggableFormat() adds to context.doc() instead of context.documentInput(), inconsistent with the purpose of the method and every other implementation in this diff. This is in test code and likely an oversight, but the inconsistency is anomalous.

The table above displays the top 10 most important findings.

Total: 1 | Critical: 0 | High: 0 | Medium: 0 | Low: 1

Pull Requests Author(s): Please update your Pull Request according to the report above.

Repository Maintainer(s): You can bypass diff analyzer by adding label skip-diff-analyzer after reviewing the changes carefully, then re-run failed actions. To re-enable the analyzer, remove the label, then re-run all actions.

⚠️ Note: The Code-Diff-Analyzer helps protect against potentially harmful code patterns. Please ensure you have thoroughly reviewed the changes beforehand.

Thanks.

github-actions · 2026-03-26T07:47:05Z

Failed to generate code suggestions for PR

github-actions · 2026-03-26T07:53:56Z

Failed to generate code suggestions for PR

github-actions · 2026-03-26T09:11:50Z

✅ Gradle check result for 93426fa: SUCCESS

codecov · 2026-03-26T09:12:29Z

Codecov Report

❌ Patch coverage is 89.36170% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.12%. Comparing base (726ee70) to head (1ae69d7).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...mapper/annotatedtext/AnnotatedTextFieldMapper.java	0.00%	9 Missing ⚠️
.../opensearch/index/mapper/size/SizeFieldMapper.java	0.00%	3 Missing ⚠️
...rg/opensearch/join/mapper/ParentIdFieldMapper.java	66.66%	1 Missing and 1 partial ⚠️
...h/index/mapper/ICUCollationKeywordFieldMapper.java	88.88%	0 Missing and 2 partials ⚠️
...earch/index/mapper/murmur3/Murmur3FieldMapper.java	75.00%	1 Missing and 1 partial ⚠️
...opensearch/index/mapper/FlatObjectFieldMapper.java	92.00%	1 Missing and 1 partial ⚠️
...va/org/opensearch/index/mapper/HllFieldMapper.java	86.66%	1 Missing and 1 partial ⚠️
...g/opensearch/index/mapper/WildcardFieldMapper.java	83.33%	1 Missing and 1 partial ⚠️
...pensearch/index/mapper/ScaledFloatFieldMapper.java	94.44%	0 Missing and 1 partial ⚠️
...earch/index/mapper/SearchAsYouTypeFieldMapper.java	88.88%	0 Missing and 1 partial ⚠️
... and 4 more

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #21001      +/-   ##
============================================
- Coverage     73.14%   73.12%   -0.03%     
+ Complexity    73289    73285       -4     
============================================
  Files          5969     5969              
  Lines        334584   334765     +181     
  Branches      48176    48207      +31     
============================================
+ Hits         244727   244789      +62     
- Misses        70324    70368      +44     
- Partials      19533    19608      +75

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2026-04-08T08:57:01Z

Persistent review updated to latest commit 11179ad

github-actions · 2026-04-08T09:46:12Z

❌ Gradle check result for 11179ad: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2026-04-08T09:51:06Z

Persistent review updated to latest commit 11179ad

github-actions · 2026-04-08T10:36:05Z

Persistent review updated to latest commit c6a5620

github-actions · 2026-04-08T12:08:03Z

✅ Gradle check result for c6a5620: SUCCESS

mgodwan · 2026-04-09T12:02:59Z

Could you please resolve conflicts on this? Rest LGTM

1. Introducing FeatureFlag and Index Setting for pluggable dataformat feature. 2. Updating the FieldMappers to insert fields in DocumentInput for Multi Format Engine Signed-off-by: Sagar Darji <darsaga@amazon.com> # Conflicts: # CHANGELOG.md

…Mapper class Signed-off-by: Sagar Darji <darsaga@amazon.com>

…gableFormat as default rather than abstract Signed-off-by: Sagar Darji <darsaga@amazon.com>

github-actions · 2026-04-09T12:41:39Z

Persistent review updated to latest commit 1ae69d7

github-actions · 2026-04-09T14:13:04Z

❕ Gradle check result for 1ae69d7: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

…#21001) * Following changes are part of this PR: 1. Introducing FeatureFlag and Index Setting for pluggable dataformat feature. 2. Updating the FieldMappers to insert fields in DocumentInput for Multi Format Engine Signed-off-by: Sagar Darji <darsaga@amazon.com> # Conflicts: # CHANGELOG.md * Addressing comment for moving the pluggable dataformat check to FieldMapper class Signed-off-by: Sagar Darji <darsaga@amazon.com> * Addressing comments for test cases and making parseCreateFieldForPluggableFormat as default rather than abstract Signed-off-by: Sagar Darji <darsaga@amazon.com> --------- Signed-off-by: Sagar Darji <darsaga@amazon.com> Co-authored-by: Sagar Darji <darsaga@amazon.com> Signed-off-by: Aparajita Pandey <aparajita31pandey@gmail.com>

…#21001) * Following changes are part of this PR: 1. Introducing FeatureFlag and Index Setting for pluggable dataformat feature. 2. Updating the FieldMappers to insert fields in DocumentInput for Multi Format Engine Signed-off-by: Sagar Darji <darsaga@amazon.com> # Conflicts: # CHANGELOG.md * Addressing comment for moving the pluggable dataformat check to FieldMapper class Signed-off-by: Sagar Darji <darsaga@amazon.com> * Addressing comments for test cases and making parseCreateFieldForPluggableFormat as default rather than abstract Signed-off-by: Sagar Darji <darsaga@amazon.com> --------- Signed-off-by: Sagar Darji <darsaga@amazon.com> Co-authored-by: Sagar Darji <darsaga@amazon.com>

darjisagar7 requested review from a team, Bukhtawar, CEHENKLE, Rishikesh1159, anasalkouz, andrross, ashking94, cwperks, dbwiddis, gbbafna, jed326, kotwanikunal, mch2, msfroh, owaiskazi19, reta, sachinpkale, saratvemulapalli, shwetathareja and sohami as code owners March 26, 2026 05:30

darjisagar7 marked this pull request as draft March 26, 2026 05:30

darjisagar7 force-pushed the DocumentInput branch from 7f26163 to 80f72cd Compare March 26, 2026 07:45

darjisagar7 force-pushed the DocumentInput branch from 80f72cd to 93426fa Compare March 26, 2026 07:52

darjisagar7 closed this Apr 8, 2026

darjisagar7 reopened this Apr 8, 2026

mgodwan reviewed Apr 8, 2026

View reviewed changes

Comment thread modules/mapper-extras/src/main/java/org/opensearch/index/mapper/RankFeatureFieldMapper.java Outdated

darjisagar7 force-pushed the DocumentInput branch from 11179ad to c6a5620 Compare April 8, 2026 10:33

mgodwan approved these changes Apr 9, 2026

View reviewed changes

Sagar Darji added 3 commits April 9, 2026 18:06

Following changes are part of this PR:

2e32785

1. Introducing FeatureFlag and Index Setting for pluggable dataformat feature. 2. Updating the FieldMappers to insert fields in DocumentInput for Multi Format Engine Signed-off-by: Sagar Darji <darsaga@amazon.com> # Conflicts: # CHANGELOG.md

Addressing comment for moving the pluggable dataformat check to Field…

c6653b9

…Mapper class Signed-off-by: Sagar Darji <darsaga@amazon.com>

Addressing comments for test cases and making parseCreateFieldForPlug…

1ae69d7

…gableFormat as default rather than abstract Signed-off-by: Sagar Darji <darsaga@amazon.com>

darjisagar7 force-pushed the DocumentInput branch from c6a5620 to 1ae69d7 Compare April 9, 2026 12:39

mgodwan approved these changes Apr 9, 2026

View reviewed changes

mgodwan merged commit 9189678 into opensearch-project:main Apr 9, 2026
15 of 16 checks passed

mgodwan mentioned this pull request Apr 10, 2026

[META] OpenSearch Pluggable Indexing #20876

Open

31 tasks

Uh oh!

Conversation

darjisagar7 commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions Bot commented Mar 26, 2026

Uh oh!

github-actions Bot commented Mar 26, 2026

Uh oh!

github-actions Bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Code Analyzer ❗

Uh oh!

github-actions Bot commented Mar 26, 2026

Uh oh!

github-actions Bot commented Mar 26, 2026

Uh oh!

github-actions Bot commented Mar 26, 2026

Uh oh!

codecov Bot commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions Bot commented Apr 8, 2026

Uh oh!

github-actions Bot commented Apr 8, 2026

Uh oh!

github-actions Bot commented Apr 8, 2026

Uh oh!

Uh oh!

github-actions Bot commented Apr 8, 2026

Uh oh!

github-actions Bot commented Apr 8, 2026

Uh oh!

mgodwan commented Apr 9, 2026

Uh oh!

github-actions Bot commented Apr 9, 2026

Uh oh!

github-actions Bot commented Apr 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

darjisagar7 commented Mar 26, 2026 •

edited

Loading

github-actions Bot commented Mar 26, 2026 •

edited

Loading

codecov Bot commented Mar 26, 2026 •

edited

Loading