Skip to content

Create tasks all at once on first assignment of HashDistributionSplitAssigner#17596

Merged
losipiuk merged 2 commits intotrinodb:masterfrom
linzebing:setNoMorePartitions-early
May 26, 2023
Merged

Create tasks all at once on first assignment of HashDistributionSplitAssigner#17596
losipiuk merged 2 commits intotrinodb:masterfrom
linzebing:setNoMorePartitions-early

Conversation

@linzebing
Copy link
Copy Markdown
Member

@linzebing linzebing commented May 22, 2023

Description

Instead of lazily creating tasks, we create tasks all at once on first assignment of HashDistributionSplitAssigner, and also trigger setNoMorePartitions early

Benchmark on tpcds-sf1000: wall time -5.98%, cpu time +2.6%

Additional context and related issues

Release notes

(x) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label May 22, 2023
@linzebing linzebing force-pushed the setNoMorePartitions-early branch from 48a1408 to 156660f Compare May 22, 2023 20:32
@linzebing linzebing requested review from arhimondr and losipiuk May 22, 2023 20:32
Copy link
Copy Markdown
Member

@losipiuk losipiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@linzebing linzebing force-pushed the setNoMorePartitions-early branch 2 times, most recently from b6ade9e to 156660f Compare May 24, 2023 16:41
@linzebing linzebing requested a review from mwd410 May 24, 2023 20:06
@losipiuk losipiuk merged commit fbdd50c into trinodb:master May 26, 2023
@github-actions github-actions bot added this to the 419 milestone May 26, 2023
@linzebing linzebing deleted the setNoMorePartitions-early branch May 26, 2023 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants