Skip to content

Conversation

@q-andy
Copy link
Contributor

@q-andy q-andy commented Jun 12, 2025

Description

Fixes compilation error from core 3.1

With opensearch-project/OpenSearch#18277 in core, we introduce a new parameter to the constructor of IngestDocumentWrapper. AbstractBatchingProcessor interacts with IngestDocumentWrappers directly so we need to update the constructor usages to fix compilation error in #1382 1382

Related Issues

n/a

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@q-andy q-andy marked this pull request as ready for review June 12, 2025 20:52
@q-andy q-andy closed this Jun 12, 2025
@q-andy q-andy reopened this Jun 12, 2025
@q-andy q-andy changed the title Fix ingest document wrapper constructors in unit tests Fix ingest document wrapper constructor compilation errors in unit tests Jun 12, 2025
@martin-gaievski
Copy link
Member

Looks good, let's wait for CI action to pass.
Btw, @q-andy what does the value 0 mean, is it a order in which objects will be retrieved?

@q-andy
Copy link
Contributor Author

q-andy commented Jun 12, 2025

IngestDocumentWrapper is a class used by core during bulk operations through bulk API. When users pass in a bulk request it can contain multiple index/update operations in order, the slot is used to map pipeline executions/document level processor executions to their corresponding operation in the user's bulk requests so errors will be surfaced correctly.

Update operations also use IngestDocumentWrapper and can have up to 2 separate ingest documents associated with a single operation (the regular doc and the upsert doc). So having a single slot isn't enough to disambiguate between different index requests anymore. Before this wasn't an issue because operations <-> ingest documents were one-to-one mapping; for update operations, we only executed pipeline on one ingest docs. But with system ingest pipelines we want to execute pipelines on both in certain cases.

To do this, in opensearch-project/OpenSearch#18277 we introduce a new childSlot to map multiple ingest documents in an update request back to the original slot in the user's requests instead, so we can surface exceptions correctly in the many-to-many case

In this case 0 just means that the document being processed is either an index request, or the doc of an update request (so not an upsert). This isn't relevant at a processor level though.

@codecov
Copy link

codecov bot commented Jun 12, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 0.00%. Comparing base (dbe2c59) to head (237bc2b).
Report is 1 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #1383       +/-   ##
============================================
- Coverage     79.80%       0   -79.81%     
============================================
  Files           159       0      -159     
  Lines          8291       0     -8291     
  Branches       1335       0     -1335     
============================================
- Hits           6617       0     -6617     
+ Misses         1161       0     -1161     
+ Partials        513       0      -513     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@martin-gaievski martin-gaievski merged commit 4cd9220 into opensearch-project:main Jun 12, 2025
93 of 160 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jun 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants