Run VStream copy only when VGTID requires it, use TablesToCopy in those cases#18938
Merged
mattlord merged 1 commit intovitessio:mainfrom Nov 23, 2025
Merged
Conversation
…se cases Signed-off-by: twthorn <thomaswilliamthornton@gmail.com>
Contributor
Review ChecklistHello reviewers! 👋 Please follow this checklist when reviewing this Pull Request. General
Tests
Documentation
New flags
If a workflow is added or modified:
Backward compatibility
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #18938 +/- ##
==========================================
+ Coverage 69.77% 69.78% +0.01%
==========================================
Files 1608 1608
Lines 214842 214865 +23
==========================================
+ Hits 149899 149945 +46
+ Misses 64943 64920 -23 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
mattlord
approved these changes
Nov 23, 2025
Member
|
Thanks, @twthorn ! ❤️ |
beingnoble03
approved these changes
Nov 23, 2025
Member
beingnoble03
left a comment
There was a problem hiding this comment.
LGTM! Thanks for fixing this!
vitess-bot
pushed a commit
that referenced
this pull request
Nov 23, 2025
…se cases (#18938) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com>
twthorn
added a commit
to slackhq/vitess
that referenced
this pull request
Nov 25, 2025
…se cases (vitessio#18938) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com>
5 tasks
tanjinx
pushed a commit
to slackhq/vitess
that referenced
this pull request
Dec 2, 2025
…se cases (vitessio#18938) (#755) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com>
siddharth16396
pushed a commit
to siddharth16396/postpone-complete
that referenced
this pull request
Dec 3, 2025
…se cases (vitessio#18938) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Signed-off-by: siddharth16396 <siddharth16396@gmail.com>
twthorn
added a commit
to slackhq/vitess
that referenced
this pull request
Jan 28, 2026
…se cases (vitessio#18938) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com>
twthorn
added a commit
to slackhq/vitess
that referenced
this pull request
Jan 28, 2026
…se cases (vitessio#18938) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com>
5 tasks
twthorn
added a commit
to slackhq/vitess
that referenced
this pull request
Jan 29, 2026
…se cases (vitessio#18938) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com>
tanjinx
pushed a commit
to slackhq/vitess
that referenced
this pull request
Jan 30, 2026
* Improve cgroup metric management (vitessio#18791) Signed-off-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * VStream: Prevent buffering entire transactions (OOM risk), instead send chunks to client (vitessio#18849) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Run VStream copy only when VGTID requires it, use TablesToCopy in those cases (vitessio#18938) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Regenerate vtgate.pb.go proto file Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Fix tests Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Complete PR vitessio#18791 backport: Update metrics_cgroup.go Apply missing changes from PR vitessio#18791 to metrics_cgroup.go: - Replace cgroup1Manager and cgroup2Manager with single cgroupManager - Add errCgroupMetricsNotAvailable error variable - Add sync.Once for lazy initialization - Remove cgroup v1 support, only support cgroup v2 - Simplify implementation with unified cgroup manager This fixes compilation errors in metrics_cgroup_test.go. * Add missing github.com/containerd/cgroups dependency Required by metrics_cgroup.go for cgroup v1/v2 support. Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Fix cgroups import to use v3 The v1 cgroups package is incompatible with Go 1.24.10. Use cgroups/v3 consistently throughout the file. Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Fix goimports formatting Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> --------- Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com>
sbaker617
pushed a commit
to slackhq/vitess
that referenced
this pull request
Feb 5, 2026
* Improve cgroup metric management (vitessio#18791) Signed-off-by: Matt Lord <mattalord@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * VStream: Prevent buffering entire transactions (OOM risk), instead send chunks to client (vitessio#18849) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Run VStream copy only when VGTID requires it, use TablesToCopy in those cases (vitessio#18938) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Regenerate vtgate.pb.go proto file Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Fix tests Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Complete PR vitessio#18791 backport: Update metrics_cgroup.go Apply missing changes from PR vitessio#18791 to metrics_cgroup.go: - Replace cgroup1Manager and cgroup2Manager with single cgroupManager - Add errCgroupMetricsNotAvailable error variable - Add sync.Once for lazy initialization - Remove cgroup v1 support, only support cgroup v2 - Simplify implementation with unified cgroup manager This fixes compilation errors in metrics_cgroup_test.go. * Add missing github.com/containerd/cgroups dependency Required by metrics_cgroup.go for cgroup v1/v2 support. Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Fix cgroups import to use v3 The v1 cgroups package is incompatible with Go 1.24.10. Use cgroups/v3 consistently throughout the file. Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> * Fix goimports formatting Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com> --------- Signed-off-by: Thomas Thornton <thomaswilliamthornton@gmail.com>
jwangace
pushed a commit
to jwangace/vitess
that referenced
this pull request
Feb 5, 2026
…esToCopy in those cases (vitessio#18938) (vitessio#18940) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com>
jwangace
pushed a commit
to jwangace/vitess
that referenced
this pull request
Feb 5, 2026
…esToCopy in those cases (vitessio#18938) (vitessio#18940) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Signed-off-by: Jun Wang <jun.wang@demonware.net>
jwangace
pushed a commit
to jwangace/vitess
that referenced
this pull request
Feb 5, 2026
…esToCopy in those cases (vitessio#18938) (vitessio#18940) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Signed-off-by: Jun Wang <jun.wang@demonware.net>
jwangace
pushed a commit
to jwangace/vitess
that referenced
this pull request
Feb 5, 2026
…esToCopy in those cases (vitessio#18938) (vitessio#18940) Signed-off-by: twthorn <thomaswilliamthornton@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Signed-off-by: Jun Wang <jun.wang@demonware.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
We are running vstream copy whenever this TablesToCopy flag is set. It should only be used if the VGTID would trigger the copy. Otherwise, there is a race condition (clients must unset their VStream config in time to avoid repeated copies). By only running copies when the VGTID requires it, we avoid repeated copies after the operation is complete (and will ignore this flag)
Note
This is a small logic bug in #18184 . That work should result in modifying the table plan built when we specify tables to copy, but it should not cause any new/additional copy phases (than before that PR). Since that was added in v23, we should backport this fix to v23.
Related Issue(s)
Checklist
Deployment Notes
AI Disclosure