VReplication: throttle on target tablet#7364
Conversation
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
|
This PR is ready; however, it does extend #7324 so it's best to first review that. |
|
What do you think of extracting the throttleStatusOK/throttle functions and related lagThrottler/lastSuccessfulThrottleCheck properties which are repeated in tableGC/vstreamer/vreplication into its own "class"? |
Yes, let me look into it. One hidden but important difference in the two implementations is that source side calls |
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
|
@rohit-nayak-ps I will refactor this PR a bit to consolidate the calls for throttle checks; will request an additional review afterwards. |
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
|
@rohit-nayak-ps I've created a new
and which offers the functionality of checking the throttler and backing off with sleep as necessary. |
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Description
With this PR Vreplication throttles on writing to target tablet. This applied to both table-copy writes and binlog events writes.
The VReplication Engine throttles by using the standard
check, which assumes the backend is a shard's primary tablet, and which throttles by shard's lag.I'm not sure if there's a scenario where the target is a replica? If so, we can certainly also add
check-selfto the throttler's consideration.Related Issue(s)
Checklist
Deployment Notes
Impacted Areas in Vitess
Components that this PR will affect:
Because this PR extends #7324 it only makes sense to fully review it once #7324 is merged. Having said that, check out the specific changes under
go/vt/vttablet/tabletmanager/vreplication/cc @rohit-nayak-ps