Open and Close PIT for Reindexing#143248
Merged
joshua-adams-1 merged 66 commits intoelastic:mainfrom Mar 13, 2026
Merged
Conversation
…hua-adams-1/elasticsearch into reindexing-plumb-pittable-hit-source
…hua-adams-1/elasticsearch into reindexing-plumb-pittable-hit-source
…hua-adams-1/elasticsearch into reindexing-plumb-pittable-hit-source
…hua-adams-1/elasticsearch into reindexing-plumb-pittable-hit-source
As part of the new reindexing resilience work, we're migrating from using scroll search to using point-in-time (PIT). This change opens a PIT (if we're behind a feature flag) before slicing a request, and closes a PIT once the reindexing is complete. It does not pass the PIT ID into the worker actions to use, however; this will be completed in a follow-up PR. Relates: elastic/elasticsearch-team#2088
RemoteResponseParsers
…1/elasticsearch into reindexing-resilience-full
PeteGillinElastic
approved these changes
Mar 5, 2026
Member
PeteGillinElastic
left a comment
PeteGillinElastic
left a comment
There was a problem hiding this comment.
Looks good, thanks!
joshua-adams-1
added
the
auto-merge-without-approval
…1/elasticsearch into reindexing-resilience-full
joshua-adams-1
removed
the
auto-merge-without-approval
… testCancelEndpointEndToEndAsynchronously
joshua-adams-1
commented
Mar 11, 2026
| * We test synchronous (<code>?wait_for_completion=true</code>) invocation of the _cancel endpoint in this test. | ||
| */ | ||
| public void testCancelEndpointEndToEndSynchronously() throws Exception { | ||
| assumeFalse("scroll-based reindex uses a different code path", ReindexPlugin.REINDEX_PIT_SEARCH_ENABLED); |
Contributor
Author
There was a problem hiding this comment.
These tests fail because:
- On snapshot builds the
REINDEX_PIT_SEARCH_ENABLEDfeature flag is enabled - We open a PIT before we slice. This is an asynchronous operation and requires waiting
- Then we create the worker tasks
- Each worker task creates a scroll
The test flow does not account for the asynchronous waiting of the PIT. I could modify the test to account for both pit and scroll functionality, but since PIT isn't complete yet, it is easier to disable these tests unless we're running with scroll. When PIT is complete, I shall make a PIT specific version of this test. I've made a note of this in https://github.com/elastic/elasticsearch-team/issues/2088 so that I don't forget.
|
|
||
| public void testUpgradeDataStream() throws Exception { | ||
| // TODO - https://github.com/elastic/elasticsearch-team/issues/2410 | ||
| assumeFalse("PIT search cannot be used on closed indices", ReindexPlugin.REINDEX_PIT_SEARCH_ENABLED); |
Contributor
Author
There was a problem hiding this comment.
This test was failing because PIT was being used on closed indices. I sunk some time into trying to fix it, but couldn't figure it out. I therefore have disabled the test from running with PIT and will fix it as a follow up
joshua-adams-1
added a commit
to joshua-adams-1/elasticsearch
that referenced
this pull request
Mar 12, 2026
michalborek
pushed a commit
to michalborek/elasticsearch
that referenced
this pull request
Mar 23, 2026
Introduces logic to open and close a point-in-time search before slicing. Relates to: elastic/elasticsearch-team#2088
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces logic to open and close a point-in-time search before slicing.
Relates to: https://github.com/elastic/elasticsearch-team/issues/2088