Skip to content

Fix worker OOM#9949

Merged
sopel39 merged 1 commit intotrinodb:masterfrom
XuPengfei-1020:master
Jan 11, 2022
Merged

Fix worker OOM#9949
sopel39 merged 1 commit intotrinodb:masterfrom
XuPengfei-1020:master

Conversation

@XuPengfei-1020
Copy link
Copy Markdown
Member

@XuPengfei-1020 XuPengfei-1020 commented Nov 12, 2021

The io.trino.orc.reader.SliceDictionaryColumnReader#dictionaryData could retaine hundreds MB memory.
Sometime caused worker OOM.

@cla-bot cla-bot bot added the cla-signed label Nov 12, 2021
@sopel39 sopel39 requested a review from skrzypo987 November 13, 2021 21:04
skrzypo987
skrzypo987 approved these changes Nov 14, 2021
View reviewed changes
raunaqmorarka
raunaqmorarka reviewed Nov 15, 2021
View reviewed changes
@XuPengfei-1020 XuPengfei-1020 force-pushed the master branch 2 times, most recently from ccd462b to 2f150a9 Compare December 23, 2021 12:51
@sopel39 sopel39 requested a review from losipiuk December 23, 2021 14:59
@cla-bot
Copy link
Copy Markdown

cla-bot bot commented Dec 23, 2021

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: x624464767@hotmail.com.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email email@example.com
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@cla-bot cla-bot bot removed the cla-signed label Dec 23, 2021
@cla-bot cla-bot bot added the cla-signed label Dec 23, 2021
sopel39
sopel39 reviewed Jan 3, 2022
View reviewed changes
@XuPengfei-1020 XuPengfei-1020 force-pushed the master branch 2 times, most recently from cfd03d9 to d992637 Compare January 5, 2022 13:04
@XuPengfei-1020 XuPengfei-1020 requested a review from sopel39 January 5, 2022 15:02
sopel39
sopel39 approved these changes Jan 5, 2022
View reviewed changes
Copy link
Copy Markdown
Member

@sopel39 sopel39 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm % comments

@XuPengfei-1020
Account for memory usage of dictionary in SliceDictionaryColumnReader
b8aec8b 
SliceDictionaryColumnReader#dictionaryData could retain hundreds
MB memory, sometime can cause worker OOM, especially in the
situation of small worker and large concurrency.
@sopel39 sopel39 merged commit f722686 into trinodb:master Jan 11, 2022
@sopel39 sopel39 mentioned this pull request Jan 11, 2022
Closed
@github-actions github-actions bot added this to the 368 milestone Jan 11, 2022
@mosabua mosabua mentioned this pull request Jan 11, 2022
Merged
@maltesander maltesander mentioned this pull request Apr 12, 2022
Closed
7 tasks
@MnO2 MnO2 mentioned this pull request Mar 8, 2023
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sopel39 sopel39 sopel39 approved these changes

@dain dain Awaiting requested review from dain

@skrzypo987 skrzypo987 Awaiting requested review from skrzypo987

@losipiuk losipiuk Awaiting requested review from losipiuk

@martint martint Awaiting requested review from martint

@raunaqmorarka raunaqmorarka Awaiting requested review from raunaqmorarka

Assignees

No one assigned

Labels

cla-signed

Milestone

368

Development

Successfully merging this pull request may close these issues.

5 participants

@XuPengfei-1020 @martint @raunaqmorarka @sopel39 @skrzypo987