Skip to content

Conversation

@udabhas
Copy link
Contributor

@udabhas udabhas commented Nov 26, 2025

Description

Recently a change was introduced to Include MergedSegmentTransferTracker in EngineConfig, but the builder method was not updated, leading to consumers of builder method returning null when getMergedSegmentTransferTracker() is invoked. This is leading to _cat/indices api not able to fill details properly.

Currently the builder method is used by opensearch-storage-encryption plugin.

Error earlier in _cat/indices call -

2025-11-26T09:05:39,618][DEBUG][o.o.a.a.i.s.TransportIndicesStatsAction][58cb8e866462db649c30b67b616dc2f5] #[java.lang.NullPointerException]#[indices:monitor/stats] failed to execute operation for shard [[enc][0], node[ggkokXB2TOewTaL-6gu37w], [P], s[STARTED], a[id=Z_jU9sA9RwSmskOJY4_aOA]]
java.lang.NullPointerException: Cannot invoke "org.opensearch.index.merge.MergedSegmentTransferTracker.stats()" because "this.mergedSegmentTransferTracker" is null
        at org.opensearch.index.engine.OpenSearchConcurrentMergeScheduler.stats(OpenSearchConcurrentMergeScheduler.java:220)
        at org.opensearch.index.engine.InternalEngine.getMergeStats(InternalEngine.java:2635)
        at org.opensearch.index.shard.IndexShard.mergeStats(IndexShard.java:1660)
        at org.opensearch.action.admin.indices.stats.CommonStats.<init>(CommonStats.java:209)        at org.opensearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:141)        at org.opensearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:67)        at org.opensearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.onShardOperation(TransportBroadcastByNodeAction.java:504)        at org.opensearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:476)        at org.opensearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceived(TransportBroadcastByNodeAction.java:463)        at com.amazonaws.elasticsearch.ccs.CrossClusterRequestInterceptor$CrossClusterRequestHandler.messageReceived(CrossClusterRequestInterceptor.java:155)
        at org.opensearch.indexmanagement.rollup.interceptor.RollupInterceptor$interceptHandler$1.messageReceived(RollupInterceptor.kt:118)
        at org.opensearch.performanceanalyzer.transport.PerformanceAnalyzerTransportRequestHandler.messageReceived(PerformanceAnalyzerTransportRequestHandler.java:44)
        at org.opensearch.performanceanalyzer.transport.RTFPerformanceAnalyzerTransportRequestHandler.messageReceived(RTFPerformanceAnalyzerTransportRequestHandler.java:92)
        at com.amazonaws.elasticsearch.iam.IamTransportRequestHandler.messageReceived(IamTransportRequestHandler.java:69)
        at org.opensearch.wlm.WorkloadManagementTransportInterceptor$RequestHandler.messageReceived(WorkloadManagementTransportInterceptor.java:63)
        at org.opensearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:108)
        at org.opensearch.transport.TransportService$7.doRun(TransportService.java:1140)
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:984)
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

Response _cat/indices

% curl localhost:9200/_cat/indices?v
health status index                          uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .plugins-ml-config             ZLIVDoVOSZmyf3AFz5ONOQ   1   0          1            0        4kb            4kb
green  open   .plugins-ml-jobs               zAzD0FCCRlGEvTorgIldiA   1   0          1            0     12.3kb         12.3kb
green  open   enc2                           aQVN34HfQBOE_0jBkchvOw   1   0
green  open   .opendistro-job-scheduler-lock RQf2uCR1Toa4DvdH_cosog   1   0          1            0       12kb           12kb
green  open   enc                            aRRST582RVOdcRbURp891A   1   0

After adding missing MergedSegmentTransferTracker, _cat/indices working as expected

% curl localhost:9200/_cat/indices?v
health status index                          uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .plugins-ml-config             ZLIVDoVOSZmyf3AFz5ONOQ   1   0          1            0        4kb            4kb
green  open   .plugins-ml-jobs               zAzD0FCCRlGEvTorgIldiA   1   0          1            0     12.3kb         12.3kb
green  open   enc2                           aQVN34HfQBOE_0jBkchvOw   1   0          0            0       208b           208b
green  open   .opendistro-job-scheduler-lock RQf2uCR1Toa4DvdH_cosog   1   0          1            0      4.9kb          4.9kb
green  open   enc                            aRRST582RVOdcRbURp891A   1   0          0            0       208b           208b

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Summary by CodeRabbit

  • Bug Fixes
    • Fixed an issue where engine configuration objects were not properly preserving all internal state when cloned, ensuring complete configuration transfers.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Nov 26, 2025

Walkthrough

The pull request adds support for the mergedSegmentTransferTracker field to EngineConfig.Builder by introducing a new builder method and updating the toBuilder() method to propagate this configuration when cloning an EngineConfig instance. The changelog documents this fix.

Changes

Cohort / File(s) Summary
Documentation
CHANGELOG.md
Added changelog entry under Fixed (Unreleased 3.x) documenting that EngineConfig.toBuilder() now includes mergedSegmentTransferTracker
Builder Enhancement
server/src/main/java/org/opensearch/index/engine/EngineConfig.java
Added public builder method mergedSegmentTransferTracker(MergedSegmentTransferTracker) to EngineConfig.Builder; updated toBuilder() to propagate the mergedSegmentTransferTracker field in the builder chain

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

  • The changes follow an established builder pattern already present in the codebase
  • Single new method addition with straightforward field propagation
  • Verify that mergedSegmentTransferTracker is properly initialized and handled throughout the builder lifecycle
  • Ensure changelog entry accurately reflects the fix and matches code changes

Poem

🐰 A tracker field, once lost in the clone,
Now finds its way through the builder's tone.
From old config to new it shall flow,
With toBuilder's grace, the transfer will go! 🎉

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: fixing the toBuilder method in EngineConfig to include mergedSegmentTransferTracker.
Description check ✅ Passed The description provides comprehensive context, including the root cause from PR #18929, the bug impact, stack traces, before/after evidence, and affected plugins.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 468f944 and 65327b2.

📒 Files selected for processing (2)
  • CHANGELOG.md (1 hunks)
  • server/src/main/java/org/opensearch/index/engine/EngineConfig.java (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (19)
  • GitHub Check: gradle-check
  • GitHub Check: precommit (25, ubuntu-latest)
  • GitHub Check: precommit (21, windows-2025, true)
  • GitHub Check: precommit (25, ubuntu-24.04-arm)
  • GitHub Check: precommit (25, macos-15-intel)
  • GitHub Check: precommit (21, windows-latest)
  • GitHub Check: precommit (25, macos-15)
  • GitHub Check: precommit (21, macos-15-intel)
  • GitHub Check: precommit (21, ubuntu-24.04-arm)
  • GitHub Check: precommit (21, macos-15)
  • GitHub Check: precommit (21, ubuntu-latest)
  • GitHub Check: precommit (25, windows-latest)
  • GitHub Check: assemble (25, ubuntu-latest)
  • GitHub Check: assemble (25, windows-latest)
  • GitHub Check: assemble (21, windows-latest)
  • GitHub Check: assemble (25, ubuntu-24.04-arm)
  • GitHub Check: assemble (21, ubuntu-latest)
  • GitHub Check: assemble (21, ubuntu-24.04-arm)
  • GitHub Check: Analyze (java)
🔇 Additional comments (2)
server/src/main/java/org/opensearch/index/engine/EngineConfig.java (2)

357-358: Excellent fix for the toBuilder() completeness.

The addition of both clusterApplierService and mergedSegmentTransferTracker to the toBuilder() method correctly addresses the NPE issue described in the PR. Both fields were missing from the builder chain, which would cause getMergedSegmentTransferTracker() and getClusterApplierService() to return null on configs created via toBuilder().

The fix ensures that plugins like opensearch-storage-encryption that rely on the builder pattern will receive fully populated configuration objects.


826-829: Builder method follows the established pattern.

The mergedSegmentTransferTracker() builder method correctly follows the standard builder pattern used throughout this class, enabling method chaining and proper field assignment.

@github-actions
Copy link
Contributor

✅ Gradle check result for 65327b2: SUCCESS

@codecov
Copy link

codecov bot commented Nov 26, 2025

Codecov Report

❌ Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.33%. Comparing base (97d3864) to head (65327b2).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
...java/org/opensearch/index/engine/EngineConfig.java 0.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #20105      +/-   ##
============================================
- Coverage     73.33%   73.33%   -0.01%     
- Complexity    71679    71708      +29     
============================================
  Files          5790     5790              
  Lines        327549   327666     +117     
  Branches      47181    47205      +24     
============================================
+ Hits         240217   240279      +62     
- Misses        68080    68123      +43     
- Partials      19252    19264      +12     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cwperks cwperks merged commit 7866b85 into opensearch-project:main Nov 26, 2025
34 of 35 checks passed
kkewwei pushed a commit to kkewwei/OpenSearch that referenced this pull request Nov 28, 2025
rgsriram pushed a commit to rgsriram/OpenSearch that referenced this pull request Dec 5, 2025
liuguoqingfz pushed a commit to liuguoqingfz/OpenSearch that referenced this pull request Dec 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants