Add Channel Factory parameter to Translog #18918

RajatGupta02 · 2025-08-05T08:33:39Z

Description

Adds a channel factory parameter to translog constructor which was earlier always defaulted to FileChannel::open. This enables us to pass our own channel factory which was not possible by overriding otherwise.

Related Issues

Add translog encryption opensearch-storage-encryption#39

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

RajatGupta02 · 2025-08-05T08:48:28Z

cc: @andrross @sachinpkale @kumargu @cwperks

github-actions · 2025-08-05T09:35:57Z

❌ Gradle check result for a311a41: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-05T12:13:02Z

❌ Gradle check result for 5bb9f00: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

cwperks · 2025-08-06T03:05:11Z

@RajatGupta02 please add a CHANGELOG entry for this. Please note that the CHANGELOG may be reset tomorrow post code-freeze.

github-actions · 2025-08-06T04:17:12Z

❕ Gradle check result for 5bb9f00: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2025-08-06T04:17:35Z

Codecov Report

❌ Patch coverage is 57.14286% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.93%. Comparing base (669ec8d) to head (4c8df69).
⚠️ Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
...a/org/opensearch/index/translog/LocalTranslog.java	0.00%	2 Missing ⚠️
...n/java/org/opensearch/index/translog/Translog.java	80.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18918      +/-   ##
============================================
- Coverage     72.96%   72.93%   -0.03%     
- Complexity    69486    69512      +26     
============================================
  Files          5648     5648              
  Lines        319329   319334       +5     
  Branches      46194    46195       +1     
============================================
- Hits         232993   232922      -71     
- Misses        67516    67638     +122     
+ Partials      18820    18774      -46

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-08-06T07:43:03Z

✅ Gradle check result for b258b88: SUCCESS

…the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]>

Signed-off-by: Rajat Gupta <[email protected]>

github-actions · 2025-08-26T15:44:33Z

✅ Gradle check result for 6a3eced: SUCCESS

Signed-off-by: Craig Perkins <[email protected]>

github-actions · 2025-08-26T17:45:37Z

❌ Gradle check result for 51b082c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/index/translog/Translog.java

Signed-off-by: Rajat Gupta <[email protected]>

github-actions · 2025-08-26T18:57:55Z

❌ Gradle check result for 4c8df69: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-27T15:47:09Z

✅ Gradle check result for 4c8df69: SUCCESS

* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]> Signed-off-by: Ankit Jain <[email protected]>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <[email protected]> * Fix tests Signed-off-by: Rajat Gupta <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Fix conflicts Signed-off-by: Rajat Gupta <[email protected]> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <[email protected]> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <[email protected]> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <[email protected]> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <[email protected]> * Add to CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <[email protected]> * Switch to CompletionStage Signed-off-by: Craig Perkins <[email protected]> * Update CHANGELOG entry Signed-off-by: Craig Perkins <[email protected]> --------- Signed-off-by: Craig Perkins <[email protected]> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> * Add Changelog entry Signed-off-by: Rajat Gupta <[email protected]> * Add secondary constructor Signed-off-by: Rajat Gupta <[email protected]> * Modify changelog Signed-off-by: Rajat Gupta <[email protected]> * Update changelog Signed-off-by: Rajat Gupta <[email protected]> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <[email protected]> --------- Signed-off-by: Rajat Gupta <[email protected]> Signed-off-by: Atri Sharma <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: Craig Perkins <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Andre van de Ven <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]> Co-authored-by: Rajat Gupta <[email protected]> Co-authored-by: Atri Sharma <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: Craig Perkins <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Andre van de Ven <[email protected]> Co-authored-by: Anurag Rai <[email protected]>

RajatGupta02 requested review from a team, Bukhtawar, CEHENKLE, Rishikesh1159, VachaShah, anasalkouz, andrross, ashking94, cwperks, dbwiddis, gbbafna, jed326, kotwanikunal, mch2, msfroh, owaiskazi19, reta, sachinpkale, saratvemulapalli, shwetathareja and sohami as code owners August 5, 2025 08:33

opensearch-ci-bot mentioned this pull request Aug 5, 2025

[AUTOCUT] Gradle Check Flaky Test Report for RemoteFsTimestampAwareTranslogTests #15818

Open

opensearch-ci-bot mentioned this pull request Aug 5, 2025

[AUTOCUT] Gradle Check Flaky Test Report for RemoteRestoreSnapshotIT #14324

Open

anuragrai16 and others added 4 commits August 26, 2025 19:40

IllegalArgumentException when scroll ID has a node no longer part of …

d05e24c

…the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <[email protected]> Signed-off-by: Anurag Rai <[email protected]>

Add Changelog entry

a69400d

Signed-off-by: Rajat Gupta <[email protected]>

Add secondary constructor

bafa5de

Signed-off-by: Rajat Gupta <[email protected]>

Modify changelog

83b5bb5

Signed-off-by: Rajat Gupta <[email protected]>

RajatGupta02 force-pushed the translog_overload branch from 282df68 to 83b5bb5 Compare August 26, 2025 14:11

Update changelog

6a3eced

Signed-off-by: Rajat Gupta <[email protected]>

Merge branch 'main' into translog_overload

51b082c

Signed-off-by: Craig Perkins <[email protected]>

cwperks reviewed Aug 26, 2025

View reviewed changes

server/src/main/java/org/opensearch/index/translog/Translog.java Show resolved Hide resolved

Add another constructor to fix breaking change check

4c8df69

Signed-off-by: Rajat Gupta <[email protected]>

opensearch-ci-bot mentioned this pull request Aug 26, 2025

[AUTOCUT] Gradle Check Flaky Test Report for SearchPhaseControllerTests #19094

Open

opensearch-ci-bot mentioned this pull request Aug 24, 2025

[AUTOCUT] Gradle Check Flaky Test Report for DecommissionControllerTests #15833

Open

cwperks approved these changes Aug 27, 2025

View reviewed changes

cwperks merged commit 981127d into opensearch-project:main Aug 27, 2025
31 of 34 checks passed

This was referenced Aug 27, 2025

[AUTOCUT] Gradle Check Flaky Test Report for ResourceAwareTasksTests #14293

Open

[AUTOCUT] Gradle Check Flaky Test Report for StreamStringTermsAggregatorTests #18956

Closed

[AUTOCUT] Gradle Check Flaky Test Report for LocalTranslogTests #19168

Closed

RajatGupta02 mentioned this pull request Sep 15, 2025

Backport to 3.1 for channelFactory param in translog #19288

Closed

3 tasks

BrewTestBot mentioned this pull request Oct 15, 2025

opensearch 3.3.0 Homebrew/homebrew-core#249627

Merged

Add Channel Factory parameter to Translog #18918

Add Channel Factory parameter to Translog #18918

Uh oh!

Conversation

RajatGupta02 commented Aug 5, 2025

Description

Related Issues

Check List

Uh oh!

RajatGupta02 commented Aug 5, 2025

Uh oh!

github-actions bot commented Aug 5, 2025

Uh oh!

github-actions bot commented Aug 5, 2025

Uh oh!

cwperks commented Aug 6, 2025

Uh oh!

github-actions bot commented Aug 6, 2025

Uh oh!

codecov bot commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Aug 6, 2025

Uh oh!

github-actions bot commented Aug 26, 2025

Uh oh!

github-actions bot commented Aug 26, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 26, 2025

Uh oh!

github-actions bot commented Aug 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov bot commented Aug 6, 2025 •

edited

Loading