Skip to content

Add legacy note for spill to disk#22843

Merged
mosabua merged 1 commit intotrinodb:masterfrom
simpligility:spill
Jul 26, 2024
Merged

Add legacy note for spill to disk#22843
mosabua merged 1 commit intotrinodb:masterfrom
simpligility:spill

Conversation

@mosabua
Copy link
Copy Markdown
Member

@mosabua mosabua commented Jul 26, 2024

Description

This keeps coming up... might as well make it official.

Additional context and related issues

https://trinodb.slack.com/archives/CLEDR9V7G/p1722011311628259

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

@mosabua mosabua requested review from losipiuk and wendigo July 26, 2024 20:49
@cla-bot cla-bot bot added the cla-signed label Jul 26, 2024

:::{warning}
The spill to disk feature and implementation are a legacy functionality of
Trino. Consider using [](/admin/fault-tolerant-execution) with the `task` retry
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTE is not really a replacement but solves the memory issue that spilling attempts to solve.

@mosabua mosabua requested a review from electrum July 26, 2024 20:54
@github-actions github-actions bot added the docs label Jul 26, 2024
@wendigo
Copy link
Copy Markdown
Contributor

wendigo commented Jul 26, 2024

@mosabua yeah, and we should probably start removing spilling logic as it's complex and broken anyway

@mosabua
Copy link
Copy Markdown
Member Author

mosabua commented Jul 26, 2024

@mosabua yeah, and we should probably start removing spilling logic as it's complex and broken anyway

Sounds good. Wdyt @martint @dain @electrum - maybe we should create a roadmap issue and proceed?

@mosabua mosabua mentioned this pull request Jul 26, 2024
6 tasks
@mosabua
Copy link
Copy Markdown
Member Author

mosabua commented Jul 26, 2024

Here is the roadmap item - #22845

@mosabua mosabua merged commit 468cdc0 into trinodb:master Jul 26, 2024
@mosabua mosabua deleted the spill branch July 26, 2024 23:14
@github-actions github-actions bot added this to the 454 milestone Jul 26, 2024
@losipiuk
Copy link
Copy Markdown
Member

losipiuk commented Jul 27, 2024

@mosabua yeah, and we should probably start removing spilling logic as it's complex and broken anyway

complex - yeah. Broken????

We wanted to remove it from join some time ago - but athena is heavy user so we left forked operator.

cc: @pettyjamesm

@wendigo
Copy link
Copy Markdown
Contributor

wendigo commented Jul 27, 2024

@losipiuk i recall some user reported issues that we left unanswered as we don't want to invest into spilling anymore

@sopel39
Copy link
Copy Markdown
Member

sopel39 commented Jul 27, 2024

@losipiuk i recall some user reported issues that we left unanswered as we don't want to invest into spilling anymore

I don't think it's broken. It's also easier to deploy and has perf advantages as of now. I have second thoughts about removing it TBH.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

5 participants