Skip to content

Backport #13130: Tablet throttler: throttler-config-via-topo defaults 'true', deprecation message for old flags#13237

Merged
mattlord merged 1 commit intovitessio:release-17.0from
planetscale:release-17.0-throttler-config-default-enable
Jun 5, 2023
Merged

Backport #13130: Tablet throttler: throttler-config-via-topo defaults 'true', deprecation message for old flags#13237
mattlord merged 1 commit intovitessio:release-17.0from
planetscale:release-17.0-throttler-config-default-enable

Conversation

@shlomi-noach
Copy link
Contributor

@shlomi-noach shlomi-noach commented Jun 5, 2023

Backport of #13130

Description

  • v16 introduced a new vttablet flag: --throttler-config-via-topo, see the docs: https://vitess.io/docs/16.0/reference/features/tablet-throttler/
  • In v16 this flag defaults false, and the old per-tablet --enable-lag-throttler configuration is still supported.
  • For v17, the target of this PR, this PR sets the default for --throttler-config-via-topo to true. The old configuration is still supported but if used there's a deprecation warning.
  • In v18, the old configuration & logic will be compeletely removed and it will be assumed that --throttler-config-via-topo is always true whether specified or not. The flag will issue a deprecation message.
  • In v19 we will remove the flag --throttler-config-via-topo.

We remove all references to --enable-lag-throttler in Vitess's own endtoend tests, and use the dynamic throttler config everywhere.

Related Issue(s)

Follow up to:

Checklist

@shlomi-noach @mattlord
Tablet throttler: throttler-config-via-topo defaults 'true', deprecat…
19e2021 
…ion message for old flags (vitessio#13130)

* Table throttler: --throttler-config-via-topo now defaults to 'true'

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* add deprecation message

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* endtoend tests: remove '--enable-lag-throttler' and use 'UpdateThrottlerConfig' everywhere

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* always use vtctldclient

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* use cluster.VtctldClientProcess

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* disable --throttler-config-via-topo in old throttler tests

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* Remove --throttler-config-via-topo where used, since it now defaults 'true'

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* fix vreplication cluster setup, waiting for throttler config to apply

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* changelog

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* extend throttler threshold

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* a bit more verbose

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* fixed CLI test

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* remove old '--enable-lag-throttler' flag, introduce '--heartbeat_on_demand_duration'

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* more log info in throttler.Open()

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* more logging

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* Revert to --heartbeat_enable

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* Protect throttler config change application with initMutex

And in e2e test update the throttler config on the keyspace
when it's created. Only wait for the new tablets in a shard
to have the throttler enabled when adding a Shard.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* More CI testing

Signed-off-by: Matt Lord <mattalord@gmail.com>

* CI testing cont

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Yes...

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Somebody doesn't like force pushes so msg here

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Increase on-demand heartbeat duration from 10s to 1m

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use only on-demand heartbeats everywhere

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use same throttler config everywhere

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Update all keyspaces and don't fail test on missing JSON keys

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use constant heartbeats in vrepl e2e tests

Until vitessio#13175 is
fixed.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Increase workflow command timeout

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Don't wait for throttler on non-serving primaries

Signed-off-by: Matt Lord <mattalord@gmail.com>

* vitessio#13175 is fixed, therefore re-instating on-deman heartbeats

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* Added ToC

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* Tweak comment and kick CI

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Treat isOpen as the ready/running signal.

Also align all initMutex usage.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Re-adjust comment

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Adjust CheckIsReady() to match OnlineDDL's expectation/usage

This was only using IsReady() before, now it's using
IsOpen() and IsReady().

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Get rid of log messages from SrvKeyspaceWatcher when no node/key

Signed-off-by: Matt Lord <mattalord@gmail.com>

* More corrections/tweaks

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use more convenient/clear new IsRunning function

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Revert "Use more convenient/clear new IsRunning function"

This reverts commit 9aef276 as this
change was not correct.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Further fix correct use of IsOpen(), IsRunning(), IsEnabled()

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* throttler.throttledApps cannot be nil

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* minor refactory/beautify for test

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* fix flakiness of tabletmanager_throttler_topo test by: (1) proper wait-for functions, and (2) issue different queries per goroutine

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>

* Fix typo in release notes

Signed-off-by: Matt Lord <mattalord@gmail.com>

---------

Signed-off-by: Shlomi Noach <2607934+shlomi-noach@users.noreply.github.com>
Signed-off-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: Matt Lord <mattalord@gmail.com>
@vitess-bot vitess-bot bot added NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsIssue A linked issue is missing for this Pull Request NeedsWebsiteDocsUpdate What it says labels Jun 5, 2023
@vitess-bot
Copy link
Contributor

vitess-bot bot commented Jun 5, 2023

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has a descriptive title.
  • If this is a change that users need to know about, please apply the release notes (needs details) label so that merging is blocked unless the summary release notes document is included.
  • If a test is added or modified, there should be a documentation on top of the test to explain what the expected behavior is what the test does.

If a new flag is being introduced:

  • Is it really necessary to add this flag?
  • Flag names should be clear and intuitive (as far as possible)
  • Help text should be descriptive.
  • Flag names should use dashes (-) as word separators rather than underscores (_).

If a workflow is added or modified:

  • Each item in Jobs should be named in order to mark it as required.
  • If the workflow should be required, the maintainer team should be notified.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should include a link to an issue that describes the bug.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.
  • RPC changes should be compatible with vitess-operator
  • If a flag is removed, then it should also be removed from VTop, if used there.

@github-actions github-actions bot added this to the v17.0.0 milestone Jun 5, 2023
@mattlord mattlord mentioned this pull request Jun 5, 2023
Closed
42 tasks
@shlomi-noach shlomi-noach added Backport This is a backport Component: Throttler Type: Bug Type: Enhancement Logical improvement (somewhere between a bug and feature) and removed NeedsDescriptionUpdate The description is not clear or comprehensive enough, and needs work NeedsWebsiteDocsUpdate What it says NeedsIssue A linked issue is missing for this Pull Request labels Jun 5, 2023
@mattlord mattlord merged commit 172dcfe into vitessio:release-17.0 Jun 5, 2023
@mattlord mattlord deleted the release-17.0-throttler-config-default-enable branch June 5, 2023 17:49
@jjpersch jjpersch mentioned this pull request Sep 21, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mattlord mattlord mattlord approved these changes

@harshit-gangal harshit-gangal Awaiting requested review from harshit-gangal harshit-gangal is a code owner

@systay systay Awaiting requested review from systay systay is a code owner

@rohit-nayak-ps rohit-nayak-ps Awaiting requested review from rohit-nayak-ps rohit-nayak-ps is a code owner

@deepthi deepthi Awaiting requested review from deepthi

Assignees

No one assigned

Labels

Backport This is a backport Component: Throttler Type: Bug Type: Enhancement Logical improvement (somewhere between a bug and feature)

Projects

None yet

Milestone

v17.0.0

Development

Successfully merging this pull request may close these issues.

2 participants

@shlomi-noach @mattlord