Reindex relocation: store source TaskResult at destination node by samxbr · Pull Request #145488 · elastic/elasticsearch

samxbr · 2026-04-01T20:21:43Z

In order to make reindex management API resilient to failures, the relocation destination node needs to write the relocation source task result into .tasks index. This ensures the relocated task can be found by management APIs from following the relocation chain stored in .tasks, if source node has ungracefully failed before writing the task result into .tasks.

We also need to ensure the source task does not override the result stored by destination, this will be handled in a separate PR.

elasticsearchmachine · 2026-04-01T21:50:53Z

Pinging @elastic/es-distributed (Team:Distributed)

modules/reindex/src/main/java/org/elasticsearch/reindex/Reindexer.java

modules/reindex/src/main/java/org/elasticsearch/reindex/TaskRelocatedException.java

modules/reindex/src/main/java/org/elasticsearch/reindex/Reindexer.java

szybia · 2026-04-02T17:36:24Z

just looked at overall approach, and spotted this: #145488 (comment)

so asking before proceeding since might mean a rework, unless i'm missing something

Copilot

Pull request overview

This PR improves reindex relocation resilience by ensuring the relocation destination node persists the relocation source task’s result into the .tasks system index, so reindex management APIs can still follow the relocation chain if the source node fails before writing its own result.

Changes:

Plumbs TaskResultsService into reindex transport/actions and Reindexer so the destination can write the source task result into .tasks.
Extends relocation resume payload (ResumeInfo) to carry the source TaskResult across to the destination node.
Updates tests to validate that the source task result is transmitted/stored and that management APIs continue to follow relocation chains.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
x-pack/plugin/enrich/src/main/java/org/elasticsearch/xpack/enrich/action/TransportEnrichReindexAction.java	Adds `TaskResultsService` wiring for enrich’s specialized reindex transport action.
server/src/main/java/org/elasticsearch/tasks/TaskResult.java	Adds a helper to produce a new `TaskResult` with a replaced error payload.
server/src/main/java/org/elasticsearch/index/reindex/ResumeInfo.java	Adds `sourceTaskResult` to relocation resume info and serializes it over the wire.
modules/reindex/src/main/java/org/elasticsearch/reindex/TransportReindexAction.java	Injects `TaskResultsService` and passes it into `Reindexer`.
modules/reindex/src/main/java/org/elasticsearch/reindex/Reindexer.java	Stores the source task result on the destination node before continuing execution; includes source result in resume request.
modules/reindex/src/main/java/org/elasticsearch/reindex/TaskRelocatedException.java	Introduces a constructor to populate relocation metadata directly (replaces setter-style API).
modules/reindex/src/test/java/org/elasticsearch/reindex/ReindexerTests.java	Adds unit tests for transmitting/storing the source task result and failure handling.
modules/reindex-management/src/test/java/org/elasticsearch/reindex/management/TransportGetReindexActionTests.java	Updates tests to use the new `TaskRelocatedException` constructor.
modules/reindex-management/src/internalClusterTest/java/org/elasticsearch/reindex/management/ReindexRelocationIT.java	Adds an IT intended to verify destination-side `.tasks` write preserves the relocation chain.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

server/src/main/java/org/elasticsearch/index/reindex/ResumeInfo.java

server/src/main/java/org/elasticsearch/tasks/TaskResult.java

modules/reindex/src/main/java/org/elasticsearch/reindex/TaskRelocatedException.java

...t/src/internalClusterTest/java/org/elasticsearch/reindex/management/ReindexRelocationIT.java

szybia

nothing blocking, LGTM!

i guess have a scan at comments and see whether you agree, can be done as follow-up PR too

...t/src/internalClusterTest/java/org/elasticsearch/reindex/management/ReindexRelocationIT.java

...ement/src/test/java/org/elasticsearch/reindex/management/TransportGetReindexActionTests.java

modules/reindex/src/main/java/org/elasticsearch/reindex/Reindexer.java

server/src/main/java/org/elasticsearch/index/reindex/ResumeInfo.java

...t/src/internalClusterTest/java/org/elasticsearch/reindex/management/ReindexRelocationIT.java

samxbr · 2026-04-08T16:33:02Z

nothing blocking, LGTM!

i guess have a scan at comments and see whether you agree, can be done as follow-up PR too

Thanks for the comments. I am inclined to merge this PR first since it's blocking my next PRs, in which I will also address theses comments.

…cation * upstream/main: Reindex relocation: store source TaskResult at destination node (elastic#145488) Bump versions after 9.2.8 release [CI] DLMFrozenTransitionServiceTests testCheckForFrozenIndicesReturnsEarlyWhenCapacityExhausted failing [elastic#145778] (elastic#145906) Update branches.json for 9.2.8 release ESQL: Clarify inheriting from Attributes (elastic#145898) Bump versions after 9.3.3 release Update branches.json for 9.3.3 release

* upstream/main: Mute org.elasticsearch.xpack.esql.expression.function.aggregate.FirstDocIdGroupingAggregatorFunctionTests testSimple elastic#145923 Reindex relocation: store source TaskResult at destination node (elastic#145488) Bump versions after 9.2.8 release [CI] DLMFrozenTransitionServiceTests testCheckForFrozenIndicesReturnsEarlyWhenCapacityExhausted failing [elastic#145778] (elastic#145906) Update branches.json for 9.2.8 release ESQL: Clarify inheriting from Attributes (elastic#145898) Bump versions after 9.3.3 release Update branches.json for 9.3.3 release Prune changelogs after 8.19.14 release Bump versions after 8.19.14 release Update branches.json for 8.19.14 release [ML] Call old inference API (elastic#145690) ESQL: Unmute CsvIT sumWithOverflowRow (elastic#145893) Index a document when testing runtime fields shadowing dimensions & metrics (elastic#145882) [TEST] Fix version check in testSequenceNumbersDisabled (elastic#145879) [ESQL] Per-file filter pushdown awareness (elastic#145755) Unmute testGetReindexFollowsRelocation (elastic#145841) Correctly ignore system indices when validating dot-prefixed indices (elastic#128868) [Transform] Remove tests for deleted code (elastic#145685) ESQL: Add generative tests for LIMIT BY (elastic#144238)

I merged elastic#145803 but it conflicted with elastic#145488 merged yesterday

samxbr · 2026-04-09T21:01:54Z

nothing blocking, LGTM!
i guess have a scan at comments and see whether you agree, can be done as follow-up PR too

Thanks for the comments. I am inclined to merge this PR first since it's blocking my next PRs, in which I will also address theses comments.

Follow up PR #145947 that includes improvements suggested in this PR.

samxbr added 2 commits April 1, 2026 13:27

Reindex relocation: store source task result at destination

bf6b036

unit tests

6e79ba8

elasticsearchmachine added the v9.4.0 label Apr 1, 2026

samxbr changed the title ~~Reindex/store source result~~ Reindex relocation: store source TaskResult at destination node Apr 1, 2026

samxbr added :Distributed/Reindex Issues relating to reindex that are not caused by issues further down >non-issue labels Apr 1, 2026

samxbr marked this pull request as ready for review April 1, 2026 21:50

elasticsearchmachine added the Team:Distributed Meta label for distributed team. label Apr 1, 2026

samxbr added 2 commits April 1, 2026 18:05

Add IT

744bb52

Merge branch 'main' into reindex/store-source-result

aec43af

samxbr requested review from PeteGillinElastic and szybia April 2, 2026 14:05

szybia reviewed Apr 2, 2026

View reviewed changes

samxbr added 6 commits April 7, 2026 10:22

Merge branch 'main' into reindex/store-source-result

a2ff458

address comment

c30c6b0

Merge branch 'main' into reindex/store-source-result

637166e

merge conflict

ef73b03

Merge branch 'main' into reindex/store-source-result

8f91e71

fix tests

9825226

samxbr requested a review from szybia April 7, 2026 18:35

szybia requested a review from Copilot April 8, 2026 11:24

Copilot started reviewing on behalf of szybia April 8, 2026 11:25 View session

Copilot AI reviewed Apr 8, 2026

View reviewed changes

szybia approved these changes Apr 8, 2026

View reviewed changes

samxbr merged commit 867e120 into elastic:main Apr 8, 2026
39 checks passed

joshua-adams-1 added a commit to joshua-adams-1/elasticsearch that referenced this pull request Apr 9, 2026

Fix merge compilation errors

44f0bf9

I merged elastic#145803 but it conflicted with elastic#145488 merged yesterday

joshua-adams-1 mentioned this pull request Apr 9, 2026

Fix main compilation errors #145989

Closed

samxbr mentioned this pull request Apr 9, 2026

Reindex Relocation: Prevent source task override task result #145947

Open

Conversation

samxbr commented Apr 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Apr 1, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

szybia commented Apr 2, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

szybia left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

samxbr commented Apr 8, 2026

Uh oh!

Uh oh!

samxbr commented Apr 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

samxbr commented Apr 1, 2026 •

edited

Loading