Documenting VReplication throttling#689
Merged
shlomi-noach merged 3 commits intoprodfrom Jan 28, 2021
Merged
Conversation
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
|
Deploy preview for vitess ready! Built with commit 7e407e2 |
Contributor
Author
|
Relevant doc preview: https://deploy-preview-689--vitess.netlify.app/docs/reference/vreplication/throttling/ |
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
deepthi
approved these changes
Jan 28, 2021
Collaborator
deepthi
left a comment
deepthi
left a comment
There was a problem hiding this comment.
Some typos / re-wording. Please address them before merging.
|
|
||
| ### Introduction | ||
|
|
||
| VReplication moves potentially massiva amounts of data from one place to another, whether within the same keyspace and shard or across keyspaces. It copies data of entire tables and follows up to apply ongoing changes on those tables by reading the binary logs (aka the changelog). |
Collaborator
There was a problem hiding this comment.
Suggested change
| VReplication moves potentially massiva amounts of data from one place to another, whether within the same keyspace and shard or across keyspaces. It copies data of entire tables and follows up to apply ongoing changes on those tables by reading the binary logs (aka the changelog). | |
| VReplication moves potentially massive amounts of data from one place to another, whether within the same keyspace and shard or across keyspaces. It copies data of entire tables and follows up to apply ongoing changes on those tables by reading the binary logs (aka the changelog). |
|
|
||
| This places load on both the source side (where VReplication reads data from) as well as on target side (where VReplication writes data to). | ||
|
|
||
| On the source side, VReplication reads the full content of tables. This typically means loading pages from disk contending for disk IO, and "polluting" the MySQL buffer pool. The operation competes with normal production traffic on both IO and memory resources. If the source is a replica, the operation may lead to replication lag. If on a primary, this may lead to write contention. |
Collaborator
There was a problem hiding this comment.
Suggested change
| On the source side, VReplication reads the full content of tables. This typically means loading pages from disk contending for disk IO, and "polluting" the MySQL buffer pool. The operation competes with normal production traffic on both IO and memory resources. If the source is a replica, the operation may lead to replication lag. If on a primary, this may lead to write contention. | |
| On the source side, VReplication reads the full content of tables. This typically means loading pages from disk contending for disk IO, and "polluting" the MySQL buffer pool. The operation competes with normal production traffic for both IO and memory resources. If the source is a replica, the operation may lead to replication lag. If the source is a primary, this may lead to write contention. |
|
|
||
| ### Source throttling | ||
|
|
||
| On the source side, VReplication only affects the single MySQL server it reads from, and has no impact on the overall shard. VStreamer, the source endpoint of VReplication, consults the equivalent of `/throttlet/check-self`, which looks for replication lag on the source host. |
Collaborator
There was a problem hiding this comment.
Suggested change
| On the source side, VReplication only affects the single MySQL server it reads from, and has no impact on the overall shard. VStreamer, the source endpoint of VReplication, consults the equivalent of `/throttlet/check-self`, which looks for replication lag on the source host. | |
| On the source side, VReplication only affects the single MySQL server it reads from, and has no impact on the overall shard. VStreamer, the source endpoint of VReplication, consults the equivalent of `/throttler/check-self`, which looks for replication lag on the source host. |
|
|
||
| On the source side, VReplication only affects the single MySQL server it reads from, and has no impact on the overall shard. VStreamer, the source endpoint of VReplication, consults the equivalent of `/throttlet/check-self`, which looks for replication lag on the source host. | ||
|
|
||
| As log as `check-self` fails, VStreamer will not read table data, nor will it pull events from the changelog. |
Collaborator
There was a problem hiding this comment.
Suggested change
| As log as `check-self` fails, VStreamer will not read table data, nor will it pull events from the changelog. | |
| As long as `check-self` fails, VStreamer will not read table data, nor will it pull events from the changelog. |
Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tracking issue: vitessio/vitess#7362
This documents the newly introduced VReplication throttling, per vitessio/vitess#7324 and vitessio/vitess#7319
Since those PRs are merged at this time, so should this PR be merged.