Add exponential backoff for task scheduling with task-level retries#11940
Add exponential backoff for task scheduling with task-level retries#11940losipiuk merged 2 commits intotrinodb:masterfrom
Conversation
arhimondr
left a comment
arhimondr
left a comment
There was a problem hiding this comment.
LGTM % session properties and testing
There was a problem hiding this comment.
There's RETRY_INITIAL_DELAY, RETRY_MAX_DELAY session properties. What do you think about reusing those? (maybe also re-adjust the defaults, starting with 1m and capping at 5m)?
There was a problem hiding this comment.
Good point. I started adding new properties yesterday - but I think reusing those is fine. It could be more elastic to have separate ones for case when queries with query-level and task-level retries are run in the same cluster - but it is not a production supported case right now.
There was a problem hiding this comment.
There's no session property for exponential growth base though. Do we want to have it?
losipiuk
force-pushed
the
lo/task-retries-backoff
branch
2 times, most recently
from
624ad34 to
cacdf60
Compare
|
Added tests |
core/trino-main/src/main/java/io/trino/execution/scheduler/FaultTolerantStageScheduler.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/execution/scheduler/FaultTolerantStageScheduler.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/test/java/io/trino/execution/scheduler/TestFaultTolerantStageScheduler.java
Outdated
Show resolved
Hide resolved
losipiuk
force-pushed
the
lo/task-retries-backoff
branch
from
cacdf60 to
b8c794b
Compare
|
AC and added one test |
core/trino-main/src/main/java/io/trino/execution/scheduler/FaultTolerantStageScheduler.java
Outdated
Show resolved
Hide resolved
core/trino-main/src/main/java/io/trino/execution/scheduler/FaultTolerantStageScheduler.java
Outdated
Show resolved
Hide resolved
losipiuk
force-pushed
the
lo/task-retries-backoff
branch
from
b8c794b to
66b03cb
Compare
|
added docs |
losipiuk
force-pushed
the
lo/task-retries-backoff
branch
from
66b03cb to
3216b31
Compare
|
CI: #11784 |
|
Merged. Thanks for review @linzebing @arhimondr |
3 participants
Add exponential backoff for task scheduling with task-level retries
Description
enhancement
core
Related issues, pull requests, and links
Documentation
( ) No documentation is needed.
(x) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
(x) No release notes entries required.
( ) Release notes entries required with the following suggested text: