Fix ArrayIndexOutOfBoundsException in fetch phase with partial results by reugn · Pull Request #144385 · elastic/elasticsearch

reugn · 2026-03-17T09:08:37Z

When a search fetch phase times out on a shard with allow_partial_search_results: true, the coordinating node can hit an ArrayIndexOutOfBoundsException in SearchPhaseController.getHits() or SearchPhaseController.mergeSuggest(). This is caused by:

FetchPhaseDocsIterator.iterate() — on timeout, used System.arraycopy(searchHits, 0, partial, 0, i) to build the partial result. Because docs are iterated in doc-id order but placed at their original (score-sorted) positions, this copy produced a corrupted array. It included nulls at unfilled positions and dropped valid hits stored beyond index i. Fixed by replacing the arraycopy with a stripNulls method that returns a compact array of only the successfully fetched hits.
SearchPhaseController.getHits() / mergeSuggest() — the query phase promises N docs from a shard, but the fetch phase may return fewer after a timeout. The merge loop used a counter to index into the fetch result array without checking bounds, causing ArrayIndexOutOfBoundsException. Fixed by adding a bounds check that skips entries when the counter exceeds the available hits.

Tests

FetchPhaseDocsIteratorTests.testTimeoutReturnsCompactPartialResults — verifies that iterate() returns a compact array (no nulls, shorter than input) when a timeout occurs mid-fetch with unsorted doc ids.
SearchPhaseControllerTests.testMergeWithPartialFetchResults — verifies that SearchPhaseController.merge() handles a shard returning fewer fetch hits than expected without throwing ArrayIndexOutOfBoundsException.

Fixes #140495

elasticsearchmachine · 2026-03-17T09:09:03Z

Pinging @elastic/es-search-foundations (Team:Search Foundations)

elasticsearchmachine · 2026-03-17T09:09:27Z

Hi @reugn, I've created a changelog YAML for you.

spinscale · 2026-03-20T09:16:16Z

server/src/main/java/org/elasticsearch/search/fetch/FetchPhaseDocsIterator.java

    }

+    private static SearchHit[] stripNulls(SearchHit[] searchHits) {
+        return Arrays.stream(searchHits).filter(Objects::nonNull).toArray(SearchHit[]::new);


drive by comment: Is there a certain overhead of using streams here, especially in the case when there are no nulls to strip? Would it be worth testing something like a for loop that counts null elements, and if that count is > 0, a new compact array is created, and returned?

Refactored to return the original array when there are no nulls. Streams are only used in the fallback path where reallocation is necessary.

spinscale · 2026-03-25T12:45:42Z

server/src/test/java/org/elasticsearch/search/fetch/FetchPhaseDocsIteratorTests.java

+        IndexReader reader = writer.getReader();
+        writer.close();
+
+        ContextIndexSearcher searcher = new ContextIndexSearcher(reader, null, null, new QueryCachingPolicy() {


could use a TrivialQueryCachingPolicy.NEVER here?

spinscale · 2026-03-25T12:51:56Z

server/src/main/java/org/elasticsearch/action/search/SearchPhaseController.java

                final int index = fetchResult.counterGetAndIncrement();
-                assert index < fetchResult.hits().getHits().length
-                    : "not enough hits fetched. index [" + index + "] length: " + fetchResult.hits().getHits().length;
+                if (index >= fetchResult.hits().getHits().length) {


is this codepath tested?

Yes, there is the SearchPhaseControllerTests.testMergeWithPartialFetchResults.

This is in SearchPhaseController.mergeSuggest() - when setting a breakpoint in that method I don't see it getting called when running testMergeWithPartialFetchResults - am I missing something? :-)

spinscale

LGTM, left two minor comments.

elasticsearchmachine · 2026-03-26T12:10:34Z

💚 Backport successful

Status	Branch	Result
✅	8.19
✅	9.2
✅	9.3

elastic#144385) When a search fetch phase times out on a shard with allow_partial_search_results: true, the coordinating node can hit an ArrayIndexOutOfBoundsException in SearchPhaseController.getHits() or SearchPhaseController.mergeSuggest(). This is caused by: FetchPhaseDocsIterator.iterate() — on timeout, used System.arraycopy(searchHits, 0, partial, 0, i) to build the partial result. Because docs are iterated in doc-id order but placed at their original (score-sorted) positions, this copy produced a corrupted array. It included nulls at unfilled positions and dropped valid hits stored beyond index i. Fixed by replacing the arraycopy with a stripNulls method that returns a compact array of only the successfully fetched hits. SearchPhaseController.getHits() / mergeSuggest() — the query phase promises N docs from a shard, but the fetch phase may return fewer after a timeout. The merge loop used a counter to index into the fetch result array without checking bounds, causing ArrayIndexOutOfBoundsException. Fixed by adding a bounds check that skips entries when the counter exceeds the available hits.

* upstream/main: (146 commits) Revert "[Native] Gradle-related tweaks to improve handling of the simdvec native library (elastic#144539)" Fix ArrayIndexOutOfBoundsException in fetch phase with partial results (elastic#144385) ESQL: Correctly manage NULL data type for SUM (elastic#144942) [ESQL] Fixes GroupedTopNBenchmark not executing (elastic#144944) Fix reader context leak when query response serialization fails (elastic#144708) Validate individual offset values in BULK_OFFSETS bounds checks (elastic#144643) Merge main21 source set into main in simdvec (elastic#144921) [TEST] Unmute TsidExtractingIdFieldMapperTests (elastic#144848) [Native] Gradle-related tweaks to improve handling of the simdvec native library (elastic#144539) Fix `ThreadedActionListenerTests#testRejectionHandling` (elastic#144795) Add new DLM Frozen Tier Transition execution plugin and service (elastic#144595) Prometheus: execute query_range via parsed EsqlStatement plan (elastic#144416) Investigate `testBulkIndexingRequestSplitting` failure (elastic#144766) Add test utility for wrapping directories in FilterDirectory layer (elastic#143563) Fix ES|QL decay tests with negative scale (elastic#144657) Fix circuit breaker leak in percolator query construction (elastic#144827) Use XPerFieldDocValuesFormat in AbstractTSDBSyntheticIdCodec (elastic#144744) [DOCS] Document how reindex work in CPS (elastic#144016) Fix Int4 vector library tests failing on Java 21 (elastic#144830) [DiskBBQ] Fix index sorting on flush (elastic#144938) ...

elastic#144385) When a search fetch phase times out on a shard with allow_partial_search_results: true, the coordinating node can hit an ArrayIndexOutOfBoundsException in SearchPhaseController.getHits() or SearchPhaseController.mergeSuggest(). This is caused by: FetchPhaseDocsIterator.iterate() — on timeout, used System.arraycopy(searchHits, 0, partial, 0, i) to build the partial result. Because docs are iterated in doc-id order but placed at their original (score-sorted) positions, this copy produced a corrupted array. It included nulls at unfilled positions and dropped valid hits stored beyond index i. Fixed by replacing the arraycopy with a stripNulls method that returns a compact array of only the successfully fetched hits. SearchPhaseController.getHits() / mergeSuggest() — the query phase promises N docs from a shard, but the fetch phase may return fewer after a timeout. The merge loop used a counter to index into the fetch result array without checking bounds, causing ArrayIndexOutOfBoundsException. Fixed by adding a bounds check that skips entries when the counter exceeds the available hits.

#144385) (#145002) When a search fetch phase times out on a shard with allow_partial_search_results: true, the coordinating node can hit an ArrayIndexOutOfBoundsException in SearchPhaseController.getHits() or SearchPhaseController.mergeSuggest(). This is caused by: FetchPhaseDocsIterator.iterate() — on timeout, used System.arraycopy(searchHits, 0, partial, 0, i) to build the partial result. Because docs are iterated in doc-id order but placed at their original (score-sorted) positions, this copy produced a corrupted array. It included nulls at unfilled positions and dropped valid hits stored beyond index i. Fixed by replacing the arraycopy with a stripNulls method that returns a compact array of only the successfully fetched hits. SearchPhaseController.getHits() / mergeSuggest() — the query phase promises N docs from a shard, but the fetch phase may return fewer after a timeout. The merge loop used a counter to index into the fetch result array without checking bounds, causing ArrayIndexOutOfBoundsException. Fixed by adding a bounds check that skips entries when the counter exceeds the available hits.

#144385) (#145003) When a search fetch phase times out on a shard with allow_partial_search_results: true, the coordinating node can hit an ArrayIndexOutOfBoundsException in SearchPhaseController.getHits() or SearchPhaseController.mergeSuggest(). This is caused by: FetchPhaseDocsIterator.iterate() — on timeout, used System.arraycopy(searchHits, 0, partial, 0, i) to build the partial result. Because docs are iterated in doc-id order but placed at their original (score-sorted) positions, this copy produced a corrupted array. It included nulls at unfilled positions and dropped valid hits stored beyond index i. Fixed by replacing the arraycopy with a stripNulls method that returns a compact array of only the successfully fetched hits. SearchPhaseController.getHits() / mergeSuggest() — the query phase promises N docs from a shard, but the fetch phase may return fewer after a timeout. The merge loop used a counter to index into the fetch result array without checking bounds, causing ArrayIndexOutOfBoundsException. Fixed by adding a bounds check that skips entries when the counter exceeds the available hits.

#144385) (#145004) When a search fetch phase times out on a shard with allow_partial_search_results: true, the coordinating node can hit an ArrayIndexOutOfBoundsException in SearchPhaseController.getHits() or SearchPhaseController.mergeSuggest(). This is caused by: FetchPhaseDocsIterator.iterate() — on timeout, used System.arraycopy(searchHits, 0, partial, 0, i) to build the partial result. Because docs are iterated in doc-id order but placed at their original (score-sorted) positions, this copy produced a corrupted array. It included nulls at unfilled positions and dropped valid hits stored beyond index i. Fixed by replacing the arraycopy with a stripNulls method that returns a compact array of only the successfully fetched hits. SearchPhaseController.getHits() / mergeSuggest() — the query phase promises N docs from a shard, but the fetch phase may return fewer after a timeout. The merge loop used a counter to index into the fetch result array without checking bounds, causing ArrayIndexOutOfBoundsException. Fixed by adding a bounds check that skips entries when the counter exceeds the available hits.

elastic#144385) When a search fetch phase times out on a shard with allow_partial_search_results: true, the coordinating node can hit an ArrayIndexOutOfBoundsException in SearchPhaseController.getHits() or SearchPhaseController.mergeSuggest(). This is caused by: FetchPhaseDocsIterator.iterate() — on timeout, used System.arraycopy(searchHits, 0, partial, 0, i) to build the partial result. Because docs are iterated in doc-id order but placed at their original (score-sorted) positions, this copy produced a corrupted array. It included nulls at unfilled positions and dropped valid hits stored beyond index i. Fixed by replacing the arraycopy with a stripNulls method that returns a compact array of only the successfully fetched hits. SearchPhaseController.getHits() / mergeSuggest() — the query phase promises N docs from a shard, but the fetch phase may return fewer after a timeout. The merge loop used a counter to index into the fetch result array without checking bounds, causing ArrayIndexOutOfBoundsException. Fixed by adding a bounds check that skips entries when the counter exceeds the available hits.

Fix ArrayIndexOutOfBoundsException in fetch phase with partial results

59c4075

reugn added >bug priority:normal A label for assessing bug priority to be used by ES engineers Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch :Search Foundations/Search Catch all for Search Foundations labels Mar 17, 2026

elasticsearchmachine added the v9.4.0 label Mar 17, 2026

Update docs/changelog/144385.yaml

921e355

reugn added the auto-backport Automatically create backport pull requests when merged label Mar 17, 2026

reugn requested a review from drempapis March 17, 2026 14:31

Merge branch 'main' into fix/array-index-out-of-bounds

c7d5666

spinscale reviewed Mar 20, 2026

View reviewed changes

reugn and others added 7 commits March 24, 2026 14:07

Merge branch 'main' into fix/array-index-out-of-bounds

2261654

Avoid unnecessary array copy in stripNulls when there are no nulls

8865c09

Remove extra null argument in tests

9d480b2

Merge branch 'main' into fix/array-index-out-of-bounds

03feb1f

Merge branch 'main' into fix/array-index-out-of-bounds

9b02304

Merge branch 'main' into fix/array-index-out-of-bounds

73c66a1

Merge branch 'main' into fix/array-index-out-of-bounds

6cafe8e

spinscale reviewed Mar 25, 2026

View reviewed changes

spinscale approved these changes Mar 25, 2026

View reviewed changes

reugn and others added 2 commits March 26, 2026 11:23

use TrivialQueryCachingPolicy.NEVER in tests

62c291b

Merge branch 'main' into fix/array-index-out-of-bounds

6968822

reugn added v8.19.0 v9.2.0 v9.3.0 labels Mar 26, 2026

reugn merged commit cc5c3ad into elastic:main Mar 26, 2026
36 checks passed

reugn deleted the fix/array-index-out-of-bounds branch March 26, 2026 12:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix ArrayIndexOutOfBoundsException in fetch phase with partial results#144385

Fix ArrayIndexOutOfBoundsException in fetch phase with partial results#144385
reugn merged 12 commits intoelastic:mainfrom
reugn:fix/array-index-out-of-bounds

reugn commented Mar 17, 2026

Uh oh!

elasticsearchmachine commented Mar 17, 2026

Uh oh!

elasticsearchmachine commented Mar 17, 2026

Uh oh!

spinscale Mar 20, 2026

Uh oh!

drempapis Mar 20, 2026

Uh oh!

reugn Mar 24, 2026

Uh oh!

spinscale Mar 25, 2026

Uh oh!

spinscale Mar 25, 2026

Uh oh!

reugn Mar 26, 2026

Uh oh!

spinscale Mar 26, 2026

Uh oh!

spinscale left a comment

Uh oh!

Uh oh!

elasticsearchmachine commented Mar 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

reugn commented Mar 17, 2026

Tests

Uh oh!

elasticsearchmachine commented Mar 17, 2026

Uh oh!

elasticsearchmachine commented Mar 17, 2026

Uh oh!

spinscale Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

drempapis Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

reugn Mar 24, 2026

Choose a reason for hiding this comment

Uh oh!

spinscale Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

spinscale Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

reugn Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

spinscale Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

spinscale left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elasticsearchmachine commented Mar 26, 2026

💚 Backport successful

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants