Reduce work in testBulkIndexingRequestSplitting#145188
Merged
DaveCTurner merged 2 commits intoelastic:mainfrom Mar 30, 2026
Merged
Conversation
No need to try and index 300 documents if we're splitting each one into its own bulk, and indeed it can take more than the allowed 10s to do so many separate bulks. With this change we sometimes try larger watermarks, and only ever try and index up to roughly twice as many docs as needed to trigger a split. Relates elastic#144579
DaveCTurner
added
>test
elasticsearchmachine
added
v9.3.3
Team:Distributed
Collaborator
|
Pinging @elastic/es-distributed (Team:Distributed) |
Member
Author
|
I'm leaving the logging etc. in place from #144766 for now but I'm pretty sure it's just that we were doing too many separate bulks to fit into the 10s. See e.g. this build scan result which did indeed just take a little over 10s to complete the work, across 240+ separate bulk requests. |
Member
Author
Changed my mind, I'm taking it out, see 807f0a8. If it fails again we can reinstate it, but these failures are so rare that we'd have to leave this open for months otherwise and it doesn't seem worth it. |
DaveCTurner
added
the
auto-backport
DaveCTurner
deleted the
2026/03/30/Netty4IncrementalRequestHandlingIT-testBulkIndexingRequestSplitting
branch
Collaborator
💔 Backport failed
You can use sqren/backport to manually backport by running |
DaveCTurner
added a commit
that referenced
this pull request
Mar 30, 2026
No need to try and index 300 documents if we're splitting each one into its own bulk, and indeed it can take more than the allowed 10s to do so many separate bulks. With this change we sometimes try larger watermarks, and only ever try and index up to roughly twice as many docs as needed to trigger a split. Closes #144579
DaveCTurner
added a commit
that referenced
this pull request
Mar 30, 2026
No need to try and index 300 documents if we're splitting each one into its own bulk, and indeed it can take more than the allowed 10s to do so many separate bulks. With this change we sometimes try larger watermarks, and only ever try and index up to roughly twice as many docs as needed to trigger a split. Closes #144579
Member
Author
mouhc1ne
pushed a commit
to shmuelhanoch/elasticsearch
that referenced
this pull request
Mar 31, 2026
No need to try and index 300 documents if we're splitting each one into its own bulk, and indeed it can take more than the allowed 10s to do so many separate bulks. With this change we sometimes try larger watermarks, and only ever try and index up to roughly twice as many docs as needed to trigger a split. Closes elastic#144579
ncordon
pushed a commit
to ncordon/elasticsearch
that referenced
this pull request
Apr 1, 2026
No need to try and index 300 documents if we're splitting each one into its own bulk, and indeed it can take more than the allowed 10s to do so many separate bulks. With this change we sometimes try larger watermarks, and only ever try and index up to roughly twice as many docs as needed to trigger a split. Closes elastic#144579
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No need to try and index 300 documents if we're splitting each one into
its own bulk, and indeed it can take more than the allowed 10s to do so
many separate bulks. With this change we sometimes try larger
watermarks, and only ever try and index up to roughly twice as many docs
as needed to trigger a split.
Closes #144579