Add mapper_type and mapper_settings to IngestionSource schema - #1155
Merged
karenyrx merged 6 commits intoJul 6, 2026
Conversation
Companion spec change for opensearch-project/OpenSearch#20722 and opensearch-project/OpenSearch#20729 which added configurable message mapping to pull-based ingestion. Adds: - IngestionSourceMapperType enum (default, raw_payload, field_mapping) - mapper_type property on IngestionSource - mapper_settings property on IngestionSource (additionalProperties for field_mapping keys: id_field, version_field, op_type_field, etc.) Signed-off-by: Rishab Nahata <rishab.nahata@uber.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
imRishN
requested review from
Tokesh,
VachaShah,
Xtansia,
aabeshov,
harshavamsi,
karenyrx,
lucy66hw and
sachetalva
as code owners
July 3, 2026 08:20
Signed-off-by: Rishab Nahata <rishab.nahata@uber.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| mapper_settings: | ||
| type: object | ||
| description: Configuration settings for the selected mapper type. For `field_mapping`, supports `id_field`, `version_field`, `op_type_field`, `op_type_field.delete_value`, and `op_type_field.create_value`. | ||
| additionalProperties: |
Contributor
There was a problem hiding this comment.
We can keep it same as params for flexibility based on OS setting definition.
Matches param pattern for forward compatibility with future mapper types that may require non-string values. Signed-off-by: Rishab Nahata <rishab.nahata@uber.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
varunbharadwaj
approved these changes
Jul 5, 2026
Avoid listing specific keys that may change across mapper types and versions. Signed-off-by: Rishab Nahata <rishab.nahata@uber.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
karenyrx
previously approved these changes
Jul 5, 2026
karenyrx
reviewed
Jul 5, 2026
| - kafka | ||
| - kinesis | ||
| - none | ||
| IngestionSourceMapperType: |
Collaborator
There was a problem hiding this comment.
@imRishN feel free to take a look at https://github.com/opensearch-project/opensearch-api-specification/blob/main/TESTING_GUIDE.md if unit tests can be added for these
Collaborator
There was a problem hiding this comment.
Version-gated to >=3.6 since mapper_type is introduced in that version. Uses file-based ingestion (no Kafka dependency) to validate that mapper_type and mapper_settings are accepted in index creation. Signed-off-by: Rishab Nahata <rishab.nahata@uber.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Moved index creation from prologue to a chapter so we can assert acknowledged/shards_acknowledged. The GET chapter now verifies that mapper_type and mapper_settings values are returned correctly in the index settings response. Signed-off-by: Rishab Nahata <rishab.nahata@uber.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
karenyrx
approved these changes
Jul 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
IngestionSourceMapperTypeenum with valuesdefault,raw_payload,field_mappingmapper_typeproperty toIngestionSourceschema referencing the new enummapper_settingsproperty toIngestionSourceschema as an object with string additionalProperties (for field_mapping config keys:id_field,version_field,op_type_field,op_type_field.delete_value,op_type_field.create_value)Companion Server PRs
mapper_settingssupport (merged)FieldMappingIngestionMessageMapperimplementation (in review)Test plan
🤖 Generated with Claude Code