Ensure all topo read calls consider `--topo_read_concurrency` by timvaillancourt · Pull Request #17276 · vitessio/vitess

timvaillancourt · 2024-11-23T01:49:17Z

Description

This PR implements the feature request #17275 by ensuring that all topo read calls respect the --topo_read_concurrency flag, specifically Get, GetVersion, List, ListDir operations

As discussed in #17275, there is a separate semaphore/limit for "global" and "cell" reads, ie: by default there can be up to 32 concurrent "global" topo reads and 32 concurrent reads per "local" cell

Related Issue(s)

Fixes: Feature Request: ensure topo calls use --topo_read_concurrency #17275
Sub-task of Tracking Issue: Tracking issue: improve VTOrc topo+backend efficiency #17330

Checklist

"Backport to:" labels have been added if this change should be back-ported to release branches
If this change is to be back-ported to previous releases, a justification is included in the PR description
Tests were added or are not required
Did the new or modified tests pass consistently locally and on CI?
Documentation was added or is not required

vitess-bot · 2024-11-23T01:49:19Z

codecov · 2024-11-23T02:40:47Z

Codecov Report

Attention: Patch coverage is 73.17073% with 11 lines in your changes missing coverage. Please review.

Project coverage is 67.53%. Comparing base (63dfb9e) to head (91e9640).
Report is 15 commits behind head on main.

Files with missing lines	Patch %	Lines
go/vt/topo/stats_conn.go	61.53%	10 Missing ⚠️
go/vt/vtorc/logic/tablet_discovery.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #17276      +/-   ##
==========================================
+ Coverage   67.51%   67.53%   +0.02%     
==========================================
  Files        1580     1581       +1     
  Lines      253805   253945     +140     
==========================================
+ Hits       171358   171506     +148     
+ Misses      82447    82439       -8

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

timvaillancourt · 2024-11-23T17:53:40Z

go/vt/topo/server.go

Moved + renamed from DefaultConcurrency from keyspace.go

DefaultReadConcurrency seems more accurate as this only applies to reads

go/vt/topo/tablet.go

go/vt/topo/stats_conn.go

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt · 2024-12-10T23:44:25Z

👋 @deepthi I think I'm done tweaking this PR now

We've backported this to our v19 branch and tested it in a low-volume environment - no issues noticed. Testing on a prod-like environment is to come if we'd like to wait for that

mattlord

Nice work on this @timvaillancourt !

The only non-nit that I had was related to the function timing stats. Let me know what you think. I can come back to this quickly and we can get it merged.

Thanks!

go/vt/topo/stats_conn.go

go/vt/topo/tablet.go

GuptaManan100

LGTM apart from the outstanding comments

go/vt/discovery/topology_watcher.go

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt · 2024-12-14T22:46:00Z

@mattlord thanks for the review. Responded to some comments and addressed @GuptaManan100's point

I think the outstanding topic is which approach to the operation vs. semaphore-wait stats timings. Thoughts appreciated 🙇

mattlord

Using request changes just to be sure that we resolve the stats questions before merging.

go/vt/topo/stats_conn.go

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt · 2024-12-16T18:28:34Z

I still think it's probably worth adding something to an existing unit test to confirm the new stats, but it looks good to me! Thanks again!

@mattlord no problem!

Yeah with all those defers crammed together it's easy to mis-read. I usually would add empty lines to separate "concepts" like this, but on big codebases I tend to be a chameleon and copy the existing style like I did here

I still think it's probably worth adding something to an existing unit test to confirm the new stats, but it looks good to me! Thanks again!

I agree, this is some important code. Adding some tests

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

…io#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Ensure all topo read calls consider `--topo_read_concurrency` (vitessio#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Revert "add keyrange support for vtorc clusters_to_watch (#457)" This reverts commit 45c2199. * [release-19.0] `vtorc`: require topo for `Healthy: true` in `/debug/health` (vitessio#17129) (vitessio#17351) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Manan Gupta <manan@planetscale.com> * `vtorc`: fetch all tablets from cells once + filter during refresh (vitessio#17388) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Support KeyRange in `--clusters_to_watch` flag (vitessio#17604) Signed-off-by: Manan Gupta <manan@planetscale.com> * missing func Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Add api end point to print the current database state in VTOrc (vitessio#15485) Signed-off-by: Manan Gupta <manan@planetscale.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Manan Gupta <manan@planetscale.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com>

…io#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Move to native sqlite3 queries (vitessio#17124) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com> * Improve efficiency of `vtorc` topo calls (vitessio#17071) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Matt Lord <mattalord@gmail.com> * Ensure all topo read calls consider `--topo_read_concurrency` (vitessio#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Avoid flaky topo concurrency test (vitessio#17407) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: fetch all tablets from cells once + filter during refresh (vitessio#17388) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Support KeyRange in `--clusters_to_watch` flag (vitessio#17604) Signed-off-by: Manan Gupta <manan@planetscale.com> * `vtorc`: improve handling of partial cell topo results (vitessio#17718) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Add stats for shards watched by VTOrc Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * add more tests Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * cleanup Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix ineffassign Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix test for v21 Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Use prefix in all vtorc check and recover logs (vitessio#17526) Signed-off-by: Eduardo J. Ortega U. <5791035+ejortegau@users.noreply.github.com> --------- Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Signed-off-by: Eduardo J. Ortega U. <5791035+ejortegau@users.noreply.github.com> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: Matt Lord <mattalord@gmail.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> Co-authored-by: Eduardo J. Ortega U. <5791035+ejortegau@users.noreply.github.com>

* Ensure all topo read calls consider `--topo_read_concurrency` (vitessio#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Revert "add keyrange support for vtorc clusters_to_watch (#457)" This reverts commit 45c2199. * [release-19.0] `vtorc`: require topo for `Healthy: true` in `/debug/health` (vitessio#17129) (vitessio#17351) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Manan Gupta <manan@planetscale.com> * `vtorc`: fetch all tablets from cells once + filter during refresh (vitessio#17388) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Support KeyRange in `--clusters_to_watch` flag (vitessio#17604) Signed-off-by: Manan Gupta <manan@planetscale.com> * missing func Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Add api end point to print the current database state in VTOrc (vitessio#15485) Signed-off-by: Manan Gupta <manan@planetscale.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Manan Gupta <manan@planetscale.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com>

…d Journal Events (#585) * VTGate VStream: Ensure reasonable delivery time for reshard journal event (vitessio#16639) Signed-off-by: Malcolm Akinje <malcolm.akinje@gmail.com> Signed-off-by: Malcolm Akinje <makinje@slack-corp.com> * Backport sqlparser patch for v15->v19 upgrade: 14763 Fix accepting bind variables in time related function calls (#590) * Fix accepting bind variables in time related function calls. (vitessio#14763) Signed-off-by: Manan Gupta <manan@planetscale.com> * fix test --------- Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> * Upgrade vitess addons to 0.19.8 (#591) This upgrade allows us to control whether vtorc raises problems or not via an environment variable. Signed-off-by: Eduardo J. Ortega U. <5791035+ejortegau@users.noreply.github.com> * Use prefix in all vtorc check and recover logs (vitessio#17526) (#592) This is a backport of vitessio#17526 . Original PR description below: Description This is meant to make recovery actions more easily identified from the logs. See vitessio#17465 Signed-off-by: Eduardo J. Ortega U. <5791035+ejortegau@users.noreply.github.com> * `slack-19.0`: various backports for `vtorc`, part 2 (#596) * Ensure all topo read calls consider `--topo_read_concurrency` (vitessio#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Revert "add keyrange support for vtorc clusters_to_watch (#457)" This reverts commit 45c2199. * [release-19.0] `vtorc`: require topo for `Healthy: true` in `/debug/health` (vitessio#17129) (vitessio#17351) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Manan Gupta <manan@planetscale.com> * `vtorc`: fetch all tablets from cells once + filter during refresh (vitessio#17388) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Support KeyRange in `--clusters_to_watch` flag (vitessio#17604) Signed-off-by: Manan Gupta <manan@planetscale.com> * missing func Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Add api end point to print the current database state in VTOrc (vitessio#15485) Signed-off-by: Manan Gupta <manan@planetscale.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Manan Gupta <manan@planetscale.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> * `slack-19.0`: `vtorc`: improve handling of partial cell topo results (#599) * `vtorc`: improve handling of partial cell topo results Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * add unit test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * improve test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * add comments Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * move sort to test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * goimports Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `slack-19.0`: skip tests that will fail on v15 downgrade testing (#605) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `slack-19.0`: Add stats for shards watched by VTOrc (#606) * Add stats for shards watched by VTOrc Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Use len() in make --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Add `GetServerStatus` RPC to use in PRS (vitessio#16022) (#607) Signed-off-by: Manan Gupta <manan@planetscale.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> * backport/patch connection pool bug/perf fixes (#604) * [release-19.0] smartconnpool: do not allow connections to starve (vitessio#17675) (vitessio#17683) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> * smartconnpool: Better handling for idle expiration (vitessio#17756) Signed-off-by: Vicent Marti <vmg@strn.cat> --------- Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Vicent Marti <vmg@strn.cat> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Vicent Martí <42793+vmg@users.noreply.github.com> Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com> * pool: reopen connection closed by idle timeout (vitessio#17818) (#609) Signed-off-by: Harshit Gangal <harshit@planetscale.com> Signed-off-by: Vicent Martí <42793+vmg@users.noreply.github.com> Co-authored-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: Vicent Martí <42793+vmg@users.noreply.github.com> * VReplication: Support excluding lagging tablets and use this in vstream manager (vitessio#17835) (#612) * `slack-19.0`: backport v22 VTOrc optimizations, part 2 (#613) * `vtorc`: remove duplicate instance read from backend (vitessio#17834) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: add index for `inst.ReadInstanceClusterAttributes` table scan Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * Add stats for shards watched by VTOrc, purge stale shards (vitessio#17815) (#616) * --consolidator-query-waiter-cap to set the max number of waiter for consolidated query (vitessio#17244) (#614) Signed-off-by: Jun Wang <jun.wang@demonware.net> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: jwang <121262788+jwangace@users.noreply.github.com> Co-authored-by: Jun Wang <jun.wang@demonware.net> * `slack-19.0` backport v22 `vtorc` optimizations + stats, part 3 (#618) * Remove unused code in discovery queue creation (vitessio#17515) Signed-off-by: Manan Gupta <manan@planetscale.com> * vtorc: Cleanup unused code (vitessio#15508) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> * `vtorc`: cleanup discover queue, add concurrency flag (vitessio#17825) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: add tablets watched stats Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix missing merge conflict update Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: skip unnecessary `inst.ReadTablet` in `logic.LockShard(...)` Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: use `errgroup` in keyspace/shard discovery Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix import Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix ineffassign Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * missing import Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: add stats for discovery workers Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * get count from backend Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * rm unused map Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Manan Gupta <manan@planetscale.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> * Bp pr 17558 pr 17858.slack19.0 (#615) * VReplication: Improve error handling in VTGate VStreams (vitessio#17558) Signed-off-by: Tom Thornton <thomaswilliamthornton@gmail.com> * Backport vitessio#17858 --------- Signed-off-by: Tom Thornton <thomaswilliamthornton@gmail.com> * `slack-19.0`: re-backport tweaks from vitessio#17911 (#621) * fix bug in reverse `if` Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * simplify Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * add `ReadTabletCountsByShard` test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * use map of map Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * capitalize Cell Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * gofmt lint Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix plural in names Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix releasing the global read lock when mysqlshell backup fails (vitessio#17000) (#623) Signed-off-by: Renan Rangel <rrangel@slack-corp.com> * VStream API: allow keyspace-level heartbeats to be streamed (vitessio#16593) (#620) * VStream API: allow keyspace-level heartbeats to be streamed (vitessio#16593) Signed-off-by: Malcolm Akinje <makinje@slack-corp.com> * `slack-19.0` backport v22 `vtorc` optimizations + stats, part 3 (#618) * Remove unused code in discovery queue creation (vitessio#17515) Signed-off-by: Manan Gupta <manan@planetscale.com> * vtorc: Cleanup unused code (vitessio#15508) Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> * `vtorc`: cleanup discover queue, add concurrency flag (vitessio#17825) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: add tablets watched stats Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix missing merge conflict update Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: skip unnecessary `inst.ReadTablet` in `logic.LockShard(...)` Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: use `errgroup` in keyspace/shard discovery Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix import Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix ineffassign Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * missing import Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * `vtorc`: add stats for discovery workers Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * get count from backend Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * rm unused map Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Manan Gupta <manan@planetscale.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> * Bp pr 17558 pr 17858.slack19.0 (#615) * VReplication: Improve error handling in VTGate VStreams (vitessio#17558) Signed-off-by: Tom Thornton <thomaswilliamthornton@gmail.com> * Backport vitessio#17858 --------- Signed-off-by: Tom Thornton <thomaswilliamthornton@gmail.com> * `slack-19.0`: re-backport tweaks from vitessio#17911 (#621) * fix bug in reverse `if` Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * simplify Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * add `ReadTabletCountsByShard` test Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * use map of map Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * capitalize Cell Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * gofmt lint Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> * fix plural in names Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> --------- Signed-off-by: Malcolm Akinje <makinje@slack-corp.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Tom Thornton <thomaswilliamthornton@gmail.com> Signed-off-by: Malcolm Akinje <malcolm.akinje@gmail.com> Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: Tom Thornton <thomaswilliamthornton@gmail.com> * Increase health check channel buffer (vitessio#17821) (#625) Signed-off-by: Manan Gupta <manan@planetscale.com> Signed-off-by: Malcolm Akinje <makinje@slack-corp.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> * VStream: Allow for automatic resume after Reshard across VStreams (vitessio#15393) (#627) Signed-off-by: Tanjin Xu <tanjin.xu@slack-corp.com> Co-authored-by: Matt Lord <mattalord@gmail.com> --------- Signed-off-by: Malcolm Akinje <malcolm.akinje@gmail.com> Signed-off-by: Malcolm Akinje <makinje@slack-corp.com> Signed-off-by: Manan Gupta <manan@planetscale.com> Signed-off-by: Eduardo J. Ortega U. <5791035+ejortegau@users.noreply.github.com> Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com> Signed-off-by: Dirkjan Bussink <d.bussink@gmail.com> Signed-off-by: Vicent Marti <vmg@strn.cat> Signed-off-by: Harshit Gangal <harshit@planetscale.com> Signed-off-by: Vicent Martí <42793+vmg@users.noreply.github.com> Signed-off-by: Jun Wang <jun.wang@demonware.net> Signed-off-by: Tom Thornton <thomaswilliamthornton@gmail.com> Signed-off-by: Renan Rangel <rrangel@slack-corp.com> Signed-off-by: Tanjin Xu <tanjin.xu@slack-corp.com> Co-authored-by: Tanjin Xu <109303790+tanjinx@users.noreply.github.com> Co-authored-by: Manan Gupta <35839558+GuptaManan100@users.noreply.github.com> Co-authored-by: Eduardo J. Ortega U. <5791035+ejortegau@users.noreply.github.com> Co-authored-by: Tim Vaillancourt <tim@timvaillancourt.com> Co-authored-by: vitess-bot[bot] <108069721+vitess-bot[bot]@users.noreply.github.com> Co-authored-by: Manan Gupta <manan@planetscale.com> Co-authored-by: Vicent Martí <42793+vmg@users.noreply.github.com> Co-authored-by: Harshit Gangal <harshit@planetscale.com> Co-authored-by: Tom Thornton <thomaswilliamthornton@gmail.com> Co-authored-by: jwang <121262788+jwangace@users.noreply.github.com> Co-authored-by: Jun Wang <jun.wang@demonware.net> Co-authored-by: Dirkjan Bussink <d.bussink@gmail.com> Co-authored-by: Renan Rangel <rvrangel@users.noreply.github.com> Co-authored-by: Matt Lord <mattalord@gmail.com>

timvaillancourt added Type: Enhancement Logical improvement (somewhere between a bug and feature) Type: Performance Component: VTOrc Vitess Orchestrator integration Component: VTCombo Component: VTTablet Component: VTGate labels Nov 23, 2024

timvaillancourt removed NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsBackportReason If backport labels have been applied to a PR, a justification is required labels Nov 23, 2024

github-actions bot added this to the v22.0.0 milestone Nov 23, 2024

timvaillancourt added Component: vtctl Component: Throttler labels Nov 23, 2024

timvaillancourt marked this pull request as ready for review November 23, 2024 15:23

timvaillancourt requested review from GuptaManan100, deepthi, frouioui, mattlord and shlomi-noach as code owners November 23, 2024 15:23

timvaillancourt mentioned this pull request Nov 23, 2024

Feature Request: ensure topo calls use --topo_read_concurrency #17275

Closed

timvaillancourt commented Nov 23, 2024

View reviewed changes

go/vt/topo/tablet.go Outdated Show resolved Hide resolved

timvaillancourt mentioned this pull request Dec 3, 2024

Tracking issue: improve VTOrc topo+backend efficiency #17330

Closed

timvaillancourt removed the NeedsWebsiteDocsUpdate What it says label Dec 3, 2024

timvaillancourt self-assigned this Dec 3, 2024

timvaillancourt commented Dec 10, 2024

View reviewed changes

go/vt/topo/stats_conn.go Outdated Show resolved Hide resolved

Ensure all topo read calls consider --topo_read_concurrency

2f1e599

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt force-pushed the topo_read_concurrency-refactor branch from df7655d to 2f1e599 Compare December 10, 2024 22:05

reset op start time

7ae2b14

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

deepthi approved these changes Dec 12, 2024

View reviewed changes

mattlord reviewed Dec 13, 2024

View reviewed changes

go/vt/topo/stats_conn.go Show resolved Hide resolved

go/vt/topo/stats_conn.go Show resolved Hide resolved

go/vt/topo/tablet.go Outdated Show resolved Hide resolved

GuptaManan100 approved these changes Dec 14, 2024

View reviewed changes

go/vt/discovery/topology_watcher.go Outdated Show resolved Hide resolved

remove unused concurrency field from topo watcher

2f31a57

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt requested a review from mattlord December 14, 2024 22:44

mattlord requested changes Dec 15, 2024

View reviewed changes

go/vt/topo/stats_conn.go Show resolved Hide resolved

mattlord approved these changes Dec 16, 2024

View reviewed changes

go/vt/topo/stats_conn.go Show resolved Hide resolved

simulate semaphore contention, test

176111c

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

func rename

91e9640

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt merged commit c48d761 into vitessio:main Dec 16, 2024

timvaillancourt deleted the topo_read_concurrency-refactor branch December 16, 2024 19:01

timvaillancourt added a commit to slackhq/vitess that referenced this pull request Jan 13, 2025

Ensure all topo read calls consider --topo_read_concurrency (vitess…

210c092

…io#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt added a commit to slackhq/vitess that referenced this pull request Feb 4, 2025

Ensure all topo read calls consider --topo_read_concurrency (vitess…

f8e26ca

…io#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt added a commit to slackhq/vitess that referenced this pull request Feb 4, 2025

Ensure all topo read calls consider --topo_read_concurrency (vitess…

cff08a5

…io#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt mentioned this pull request Feb 4, 2025

slack-19.0: various backports for vtorc, part 2 slackhq/vitess#596

Merged

5 tasks

timvaillancourt added a commit to slackhq/vitess that referenced this pull request Feb 19, 2025

Ensure all topo read calls consider --topo_read_concurrency (vitess…

72a2d0c

…io#17276) Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

timvaillancourt mentioned this pull request Feb 19, 2025

slack-21.0: backport v22 VTOrc fixes, part 1 slackhq/vitess#610

Merged

5 tasks

Conversation

timvaillancourt commented Nov 23, 2024 • edited by mattlord Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue(s)

Checklist

Uh oh!

vitess-bot bot commented Nov 23, 2024

Review Checklist

General

Tests

Documentation

New flags

If a workflow is added or modified:

Backward compatibility

Uh oh!

codecov bot commented Nov 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

timvaillancourt Nov 23, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

timvaillancourt commented Dec 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattlord left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

GuptaManan100 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

timvaillancourt commented Dec 14, 2024

Uh oh!

mattlord left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

timvaillancourt commented Dec 16, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

timvaillancourt commented Nov 23, 2024 •

edited by mattlord

Loading

codecov bot commented Nov 23, 2024 •

edited

Loading

timvaillancourt commented Dec 10, 2024 •

edited

Loading