Skip to content

Comments

Turn on optimized_scale_writer_producer_buffer by default#23774

Merged
jaystarshot merged 1 commit intoprestodb:masterfrom
jaystarshot:jay-writer
Oct 7, 2024
Merged

Turn on optimized_scale_writer_producer_buffer by default#23774
jaystarshot merged 1 commit intoprestodb:masterfrom
jaystarshot:jay-writer

Conversation

@jaystarshot
Copy link
Member

@jaystarshot jaystarshot commented Oct 4, 2024

Description

Found out a case where there was skew under the stage which is writing. This resulted in slow writer.

2ab2543 Presto already has a similar fix which is off by default. Turning it on helps but query is still a bit slower than no skew since the writers are scaling slower than the no skew case

trinodb/trino@4ddcb9e Trino's fix makes more sense and maybe it can be incorporated later

Motivation and Context

Impact

Test Plan

Contributor checklist

  • Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
  • PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
  • Documented new properties (with its default value), SQL syntax, functions, or other functionality.
  • If release notes are required, they follow the release notes guidelines.
  • Adequate tests were added if applicable.
  • CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

General Changes
* Improve writer scaling in skewed conditions by setting optimized_scale_writer_producer_buffer to on by default :pr:`23774`


@steveburnett
Copy link
Contributor

Nit, suggest rephrasing release note entry to follow the Order of changes in the Release Notes Guidelines. From the PR description, maybe something like this? Please revise my suggestion if you have a better way to describe it.

== RELEASE NOTES ==

General Changes
* Improve query speed by setting ``optimized_scale_writer_producer_buffer`` to ``on`` by default :pr:`23774`

Also, I found the session property optimized_scale_writer_producer_buffer and the configuration property optimized-scale-writer-producer-buffer mentioned in the 0.234 Release Notes, but I didn't find either of these elsewhere in the documentation. Consider adding doc for these.

@jaystarshot jaystarshot merged commit 152e64d into prestodb:master Oct 7, 2024
@jaystarshot jaystarshot mentioned this pull request Nov 1, 2024
25 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants