Skip to content

Keep chunked fetch Lucene I/O on search threads#1

Closed
drempapis wants to merge 4 commits intoserialize_hits_in_doc_id_orderfrom
chunk_fetch_ThrottledIterator
Closed

Keep chunked fetch Lucene I/O on search threads#1
drempapis wants to merge 4 commits intoserialize_hits_in_doc_id_orderfrom
chunk_fetch_ThrottledIterator

Conversation

@drempapis
Copy link
Copy Markdown
Owner

Closes elastic#144467

In elastic#139124, an earlier attempt to use ThrottledIterator with an executor (so chunk processing could continue on different threads while preserving strict sequencing) hit Lucene thread-affinity assertions such as

java.lang.AssertionError: Sorted doc values are only supposed to be consumed in the thread
in which they have been acquired. But was acquired in Thread[...search[T#5]...]
and consumed in Thread[...search[T#3]...].

Because of that, StreamingFetchPhaseDocsIterator was implemented to perform all Lucene I/O (document fetch + serialization) on the calling thread, while ThrottledTaskRunner with DIRECT_EXECUTOR_SERVICE was used only to throttle asynchronous network sends.

After revisiting the design, we can reintroduce ThrottledIterator for sequencing/backpressure by re-acquiring per-leaf readers at each chunk boundary via setNextReader(). This ensures that if chunk production moves between search threads, Lucene objects are acquired and consumed on the same thread for each chunk.

The changes in the pr also prevent Lucene I/O from running on transport threads during ACK-driven continuation. ThrottledIterator#onItemRelease() now dispatches run() to a provided search executor, so chunk fetch/serialization stays on search-related threads.

Dispatch is only performed when there is actual work to do, avoiding unnecessary scheduling in no-op cases.

  • permits.getAndIncrement() == 0, and
  • iterator.hasNext() is still true.

@drempapis drempapis closed this Apr 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant