Add temporal routing processors for time-based document routing #18966

atris · 2025-08-07T15:16:55Z

Implements temporal routing based on RFC #18920. Routes documents
to shards based on timestamp fields with configurable granularities
(hour, day, week, month). Includes search-time routing to direct
queries only to relevant time ranges.

TemporalRoutingProcessor for ingest pipelines
TemporalRoutingSearchProcessor for search pipelines
Optional hash bucketing for better shard distribution
ISO week format support

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check
here.

github-actions · 2025-08-07T15:49:05Z

❌ Gradle check result for af6a5c4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-07T17:37:25Z

✅ Gradle check result for 46d0f1f: SUCCESS

codecov · 2025-08-07T17:37:54Z

Codecov Report

❌ Patch coverage is 84.82143% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.87%. Comparing base (f967a72) to head (aa33ee9).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
...search/ingest/common/TemporalRoutingProcessor.java	81.52%	9 Missing and 8 partials ⚠️
...ipeline/common/TemporalRoutingSearchProcessor.java	87.02%	6 Missing and 11 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18966      +/-   ##
============================================
- Coverage     72.96%   72.87%   -0.09%     
+ Complexity    69451    69390      -61     
============================================
  Files          5645     5647       +2     
  Lines        318787   319011     +224     
  Branches      46125    46150      +25     
============================================
- Hits         232610   232487     -123     
- Misses        67358    67734     +376     
+ Partials      18819    18790      -29

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

...mmon/src/main/java/org/opensearch/search/pipeline/common/TemporalRoutingSearchProcessor.java

github-actions · 2025-08-08T15:50:17Z

❌ Gradle check result for 83d2ba4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-08T17:47:25Z

❌ Gradle check result for 83d2ba4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

andrross · 2025-08-08T18:05:57Z

@atris You can rebase and get the fix from #18971 to fix that latest test failure

Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. Signed-off-by: Atri Sharma <[email protected]>

Implements temporal routing based on RFC opensearch-project#18920. Routes documents to shards based on timestamp fields with configurable granularities (hour, day, week, month). Includes search-time routing to direct queries only to relevant time ranges. - TemporalRoutingProcessor for ingest pipelines - TemporalRoutingSearchProcessor for search pipelines - Optional hash bucketing for better shard distribution - ISO week format support - Comprehensive unit tests Signed-off-by: Atri Sharma <[email protected]>

Signed-off-by: Atri Sharma <[email protected]>

github-actions · 2025-08-08T20:02:52Z

✅ Gradle check result for 28ad439: SUCCESS

msfroh

Looks good overall.

I was talking with @varunbharadwaj, who also pointed out that the PR description is missing the DCO disclaimer: https://github.com/opensearch-project/OpenSearch/blob/main/.github/pull_request_template.md?plain=1#L20

@atris, I assume you did "Select All" -> <paste in prepared PR description>. Can you please add back the DCO disclaimer? I'm not a lawyer, but I assume it does something to shield the project from liability.

Also, per @varunbharadwaj's comment, maybe it makes sense to move all of these routing processors that you've added into a single module (maybe pipeline-processor-routing or something)? That way, we could reuse logic between the ingest and search processors, since they would all be loaded from the same classpath. It should be backward-compatible to move them to a different module, since they'll still be available in the -min distribution. I wouldn't do that in this PR, though -- I think that could be its own "cleanup" PR that comes later.

atris · 2025-08-09T15:53:54Z

Looks good overall.

I was talking with @varunbharadwaj, who also pointed out that the PR description is missing the DCO disclaimer: https://github.com/opensearch-project/OpenSearch/blob/main/.github/pull_request_template.md?plain=1#L20

@atris, I assume you did "Select All" -> <paste in prepared PR description>. Can you please add back the DCO disclaimer? I'm not a lawyer, but I assume it does something to shield the project from liability.

Also, per @varunbharadwaj's comment, maybe it makes sense to move all of these routing processors that you've added into a single module (maybe pipeline-processor-routing or something)? That way, we could reuse logic between the ingest and search processors, since they would all be loaded from the same classpath. It should be backward-compatible to move them to a different module, since they'll still be available in the -min distribution. I wouldn't do that in this PR, though -- I think that could be its own "cleanup" PR that comes later.

Thanks for catching that. I've added the DCO disclaimer back to the PR description.

Regarding moving all routing processors to a single module - that's a great idea. It would definitely help with code reuse, especially for the temporal bucket logic that's currently duplicated between the ingest and search processors. I
agree this should be done as a separate cleanup PR after this one is merged. I can create an issue to track that work if you'd like.

Signed-off-by: Atri Sharma <[email protected]>

github-actions · 2025-08-09T16:24:10Z

❌ Gradle check result for 356a5bb: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

CHANGELOG.md

Signed-off-by: Atri Sharma <[email protected]>

github-actions · 2025-08-09T20:50:24Z

❌ Gradle check result for aa33ee9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-11T06:23:06Z

❌ Gradle check result for aa33ee9: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-11T08:41:07Z

✅ Gradle check result for aa33ee9: SUCCESS

…search-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes #15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC #18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>

…search-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>

…search-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]> Signed-off-by: Ankit Jain <[email protected]>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>

…search-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>

atris requested a review from a team as a code owner August 7, 2025 15:16

varunbharadwaj reviewed Aug 7, 2025

View reviewed changes

...mmon/src/main/java/org/opensearch/search/pipeline/common/TemporalRoutingSearchProcessor.java Outdated Show resolved Hide resolved

varunbharadwaj reviewed Aug 8, 2025

View reviewed changes

...mmon/src/main/java/org/opensearch/search/pipeline/common/TemporalRoutingSearchProcessor.java Show resolved Hide resolved

atris closed this Aug 8, 2025

atris reopened this Aug 8, 2025

opensearch-ci-bot mentioned this pull request Aug 7, 2025

[AUTOCUT] Gradle Check Flaky Test Report for ClientYamlTestSuiteIT #14319

Open

atris added 5 commits August 9, 2025 00:06

Update CHANGELOG

3660f48

Signed-off-by: Atri Sharma <[email protected]>

More Javadocs

c235160

Signed-off-by: Atri Sharma <[email protected]>

Update comments

28ad439

Signed-off-by: Atri Sharma <[email protected]>

atris force-pushed the temporal_stuff branch from 83d2ba4 to 28ad439 Compare August 8, 2025 18:36

msfroh reviewed Aug 8, 2025

View reviewed changes

Merge branch 'main' into temporal_stuff

356a5bb

Signed-off-by: Atri Sharma <[email protected]>

varunbharadwaj reviewed Aug 9, 2025

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

Update CHANGELOG

aa33ee9

Signed-off-by: Atri Sharma <[email protected]>

opensearch-ci-bot mentioned this pull request Aug 9, 2025

[AUTOCUT] Gradle Check Flaky Test Report for AutoForceMergeManagerTests #18820

Closed

atris closed this Aug 11, 2025

atris reopened this Aug 11, 2025

atris closed this Aug 11, 2025

atris reopened this Aug 11, 2025

Divyaasm mentioned this pull request Aug 11, 2025

[AUTOCUT] Gradle Check Flaky Test Report for AutoForceMergeManagerTests Divyaasm/opensearch-build#2

Closed

msfroh approved these changes Aug 11, 2025

View reviewed changes

msfroh merged commit 1a53317 into opensearch-project:main Aug 11, 2025
85 of 89 checks passed

This was referenced Aug 8, 2025

[AUTOCUT] Gradle Check Flaky Test Report for FlightMetricsTests #18947

Open

[AUTOCUT] Gradle Check Flaky Test Report for WeightedRoutingIT #19028

Closed

[AUTOCUT] Gradle Check Flaky Test Report for SearchReplicaAllocationIT #19047

Open

opensearch-ci-bot mentioned this pull request Aug 25, 2025

[AUTOCUT] Gradle Check Flaky Test Report for S3BlobContainerRetriesTests #17551

Open

msfroh mentioned this pull request Oct 7, 2025

Add varunbharadwaj as a maintainer #19557

Merged

Add temporal routing processors for time-based document routing #18966

Add temporal routing processors for time-based document routing #18966

Uh oh!

Conversation

atris commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Aug 7, 2025

Uh oh!

github-actions bot commented Aug 7, 2025

Uh oh!

codecov bot commented Aug 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 8, 2025

Uh oh!

github-actions bot commented Aug 8, 2025

Uh oh!

andrross commented Aug 8, 2025

Uh oh!

github-actions bot commented Aug 8, 2025

Uh oh!

msfroh left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

atris commented Aug 9, 2025

Uh oh!

github-actions bot commented Aug 9, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 9, 2025

Uh oh!

github-actions bot commented Aug 11, 2025

Uh oh!

github-actions bot commented Aug 11, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

atris commented Aug 7, 2025 •

edited

Loading

codecov bot commented Aug 7, 2025 •

edited

Loading

msfroh left a comment •

edited

Loading