Decrease lock contention in PipelinedStageExecution#14030
Decrease lock contention in PipelinedStageExecution#14030radek-kondziolka wants to merge 1 commit intotrinodb:masterfrom
Conversation
radek-kondziolka
requested review from
arhimondr,
losipiuk,
lukasz-stec and
sopel39
and removed request for
losipiuk
|
please set the PR title so it is more eye-pleasing :) |
radek-kondziolka
changed the title
core/trino-main/src/main/java/io/trino/execution/scheduler/PipelinedStageExecution.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
this is super disturbing :) This commit increases complexity a lot (I think above my PR review confidence). I am not convinced we want that.
There was a problem hiding this comment.
Well, the most safe, readable and easy to reason about approach is just to use synchronized, I agree. But, as a side effect we can observe high contention, what we was actually did. The code was rewritten to use lock-free data structures and atomic counters. The decision to move the machine state from ANY_STATE to FLUSHING/FINISHED is basing on the one specific counter (runningTasks / finishedTasks) to make it easier to reason about.
We have a high lock contention what limits scaling capabilities. In some moment we need to do something with that. I would say that it is expected that lock-free code is more complex that the "lock-full" one.
Taking a monitor of io.trino.execution.scheduler.PipelinedStageExection in the updateTaskStatus method causes a high lock contention. Make this method lock-free.
radek-kondziolka
force-pushed
the
rk/mitigate_lock_contention_by_tracker2
branch
from
0b188b9 to
445cd35
Compare
radek-kondziolka
changed the title
radek-kondziolka
changed the title
|
Replaced with #14138 |
2 participants
Description
We observed that there is a high lock contention on
io.trino.execution.scheduler.PipelinedStageExecution's monitor.Lock contention before:
Lock contention after:
We measured the throughput (40 workers r5.4xlarge, 64 concurrent queries) before/after
Throughput (queries/h) before:
12230 queries / h
Throughput (queries/h) after:
13897 queries / h (12% difference)
Non-technical explanation
The Trino is able to process more queries within one hour.
Release notes
(* ) This is not user-visible and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text: