Skip to content

Start exchange sink fetchers concurrently#140196

Merged
dnhatn merged 3 commits intoelastic:mainfrom
dnhatn:start-exchange
Jan 7, 2026
Merged

Start exchange sink fetchers concurrently#140196
dnhatn merged 3 commits intoelastic:mainfrom
dnhatn:start-exchange

Conversation

@dnhatn
Copy link
Member

@dnhatn dnhatn commented Jan 6, 2026

We should start fetching pages from the exchange sink asynchronously for each client. However, the current code should not be an issue, as we go asynchronous when sending messages in production. Therefore, this is marked as a non-issue rather than a bug.

fetchExecutor.execute(new ActionRunnable<>(refs.acquire()) {
@Override
protected void doRun() {
var fetcher = new RemoteSinkFetcher(remoteSink, failFast, onPageFetched, listener);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was initially confused about this listener (that actually belongs to ActionRunnable#listener) vs one from argument.
I wonder if it is worth renaming argument to remoteSinkListener or this (and corresponding one from onFailure) prepend with this.listener prefix?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++ I pushed 81550f3

@dnhatn dnhatn marked this pull request as ready for review January 6, 2026 16:14
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jan 6, 2026
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@dnhatn dnhatn enabled auto-merge (squash) January 6, 2026 16:14
@dnhatn
Copy link
Member Author

dnhatn commented Jan 7, 2026

Thanks @idegtiarenko

@dnhatn dnhatn disabled auto-merge January 7, 2026 07:11
@dnhatn dnhatn merged commit 793c3c0 into elastic:main Jan 7, 2026
34 of 35 checks passed
@dnhatn dnhatn deleted the start-exchange branch January 7, 2026 07:12
ywangd pushed a commit to ywangd/elasticsearch that referenced this pull request Jan 7, 2026
We should start fetching pages from the exchange sink asynchronously for 
each client. However, the current code should not be an issue, as we go
asynchronous when sending messages in production. Therefore, this is
marked as a non-issue rather than a bug.
szybia added a commit to szybia/elasticsearch that referenced this pull request Jan 7, 2026
* upstream/main: (191 commits)
  Overall Decision for Deciders prioritizes THROTTLE (elastic#140237)
  Apply group by all logic not only to top-level aggregates (elastic#140248)
  [ES|QL] Refactor MV_UNION and MV_INTERSECTION to use shared set operation helper (elastic#139982)
  Avoid reading entire bloom filter file on reader open (elastic#139374)
  Mark bloom filter files for random access (elastic#139375)
  Ensure that the buffer used for ES93BloomFilterStoredFieldsFormat is zeroed (elastic#139034)
  Add busy assertion to avoid race condition for testStalledShardMigrationProperlyDetected (elastic#140230)
  Remove line number check for testTransitiveFindsDeepCallChain (elastic#140228)
  Allow a slight difference in rescored docs (elastic#139931)
  Mute org.elasticsearch.xpack.inference.integration.AuthorizationTaskExecutorIT testCreatesEisChatCompletion_DoesNotRemoveEndpointWhenNoLongerAuthorized elastic#138480
  Start exchange sink fetchers concurrently (elastic#140196)
  Allow allocation to replacement target node on vacate completion (elastic#140150)
  Ignore JNA cleaner threads in SecureHdfsRepositoryAnalysisRestIT (elastic#139925)
  DeterministicQueue refactor and enhancement (elastic#140151)
  Always error out if CCS expression shows up when CCS is not supported (elastic#139009)
  Use IllegalArgumentException over RepositoryException for readonly-repository checks (elastic#140200)
  Guard promql capabilities in AnalyzerTests (elastic#140232)
  [Inference API] Fix flaky AuthorizationTaskExecutorIT tests (elastic#139978)
  Cleaning up exitable vector value impls (elastic#140190)
  [Inference API] Fix auth exception listener not called bug (elastic#139966)
  ...
sidosera pushed a commit to sidosera/elasticsearch that referenced this pull request Jan 7, 2026
We should start fetching pages from the exchange sink asynchronously for 
each client. However, the current code should not be an issue, as we go
asynchronous when sending messages in production. Therefore, this is
marked as a non-issue rather than a bug.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Analytics/ES|QL AKA ESQL >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants