Skip to content

Remove v2 resharding fields#10409

Merged
deepthi merged 8 commits intovitessio:mainfrom
arvind-murty:am-delete-keyspace-fields
Jun 10, 2022
Merged

Remove v2 resharding fields#10409
deepthi merged 8 commits intovitessio:mainfrom
arvind-murty:am-delete-keyspace-fields

Conversation

@arvind-murty
Copy link
Contributor

Description

sharding_column_name and sharding_column_type are no longer used by resharding, so they can be deleted.

Related Issue(s)

Fixes #10378

Checklist

  • "Backport me!" label has been added if this change should be backported
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

@github-actions
Copy link
Contributor

github-actions bot commented Jun 1, 2022

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

  • Ensure that the Pull Request has the correct release notes label. release notes none should only be used for PRs that are so trivial that they need not be included.
  • If a new flag is being introduced, review whether it is really needed. The flag names should be clear and intuitive (as far as possible), and the flag's help should be descriptive.

Bug fixes

  • There should be at least one unit or end-to-end test.
  • The Pull Request description should either include a link to an issue that describes the bug OR an actual description of the bug and how to reproduce, along with a description of the fix.

Non-trivial changes

  • There should be some code comments as to why things are implemented the way they are.

New/Existing features

  • Should be documented, either by modifying the existing documentation or creating new documentation.
  • New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

  • Protobuf changes should be wire-compatible.
  • Changes to _vt tables and RPCs need to be backward compatible.
  • vtctl command output order should be stable and awk-able.

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
@arvind-murty arvind-murty force-pushed the am-delete-keyspace-fields branch from 80ef130 to e3acb9e Compare June 2, 2022 00:29
@deepthi
Copy link
Collaborator

deepthi commented Jun 2, 2022

@rohit-nayak-ps vtctld_web_test is failing on this PR. Do we need to regenerate the rice-box.go for these changes? Is that still possible?

2022-06-02T00:53:12.6438684Z     vtctld_web_main_test.go:481: 
2022-06-02T00:53:12.6439138Z         	Error Trace:	vtctld_web_main_test.go:481
2022-06-02T00:53:12.6439701Z         	            				vtctld_web_test.go:127
2022-06-02T00:53:12.6440023Z         	Error:      	elements differ
2022-06-02T00:53:12.6440310Z         	            	
2022-06-02T00:53:12.6440745Z         	            	extra elements in list B:
2022-06-02T00:53:12.6441187Z         	            	([]interface {}) (len=2) {
2022-06-02T00:53:12.6442053Z         	            	 (string) (len=29) "-sharding_column_name=test_id",
2022-06-02T00:53:12.6442783Z         	            	 (string) (len=28) "-sharding_column_type=UINT64"
2022-06-02T00:53:12.6443104Z         	            	}
2022-06-02T00:53:12.6443368Z         	            	
2022-06-02T00:53:12.6443622Z         	            	
2022-06-02T00:53:12.6443910Z         	            	listA:
2022-06-02T00:53:12.6444308Z         	            	([]string) (len=3) {
2022-06-02T00:53:12.6444834Z         	            	 (string) (len=14) "CreateKeyspace",
2022-06-02T00:53:12.6445445Z         	            	 (string) (len=12) "-force=false",
2022-06-02T00:53:12.6445953Z         	            	 (string) (len=14) "test_keyspace3"
2022-06-02T00:53:12.6446254Z         	            	}
2022-06-02T00:53:12.6446515Z         	            	
2022-06-02T00:53:12.6447017Z         	            	
2022-06-02T00:53:12.6447309Z         	            	listB:
2022-06-02T00:53:12.6447705Z         	            	([]string) (len=5) {
2022-06-02T00:53:12.6448204Z         	            	 (string) (len=14) "CreateKeyspace",
2022-06-02T00:53:12.6448909Z         	            	 (string) (len=29) "-sharding_column_name=test_id",
2022-06-02T00:53:12.6449626Z         	            	 (string) (len=28) "-sharding_column_type=UINT64",
2022-06-02T00:53:12.6450216Z         	            	 (string) (len=12) "-force=false",
2022-06-02T00:53:12.6450744Z         	            	 (string) (len=14) "test_keyspace3"
2022-06-02T00:53:12.6451039Z         	            	}
2022-06-02T00:53:12.6451319Z         	Test:       	TestCreateKs

…admin

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
@notfelineit
Copy link
Contributor

@arvind-murty did you run make web_build? That might help compile!

@notfelineit
Copy link
Contributor

Hey @arvind-murty, I went ahead and created a branch that removes those fields from vtctld2, you can rebase off #10459 to get those changes onto your branch!

Comment on lines -506 to -511
CreateKeyspace.Flags().StringVar(&createKeyspaceOptions.ShardingColumnName, "sharding-column-name", "", "The column name to use for sharding operations")
CreateKeyspace.Flags().Var(&createKeyspaceOptions.ShardingColumnType, "sharding-column-type", "The type of the column to use for sharding operations")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajm188 what is the right way to deprecate these options?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pflag has native support for this!! (vaguely gestures at "won't it be great to use pflag everywhere in the future?"):

CreateKeyspace.Flags().StringVar(&createKeyspaceOptions.ShardingColumnName, "sharding-column-name", ...)
CreateKeyspace.Flags().MarkDeprecated("sharding-column-name", "Keyspace.ShardingColumnName is no longer used for resharding") // feel free to tweak the deprecation message, i didn't give this much thought
// same pattern for ShardingColumnType

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I'm thinking we should let this PR delete these flags on main (to go into v15) and do a separate PR to deprecate them in v14 (on release-14.0)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense to me!

Comment on lines -1922 to -1923
shardingColumnName := subFlags.String("sharding_column_name", "", "Specifies the column to use for sharding operations")
shardingColumnType := subFlags.String("sharding_column_type", "", "Specifies the type of the column to use for sharding operations")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajm188 what is the right way to deprecate these?

Copy link
Contributor

@ajm188 ajm188 Jun 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this, we should definitely update the help message for these flags to begin with "(DEPRECATED) ", eg:

	shardingColumnName := subFlags.String("sharding_column_name", "", "(DEPRECATED) Specifies the column to use for sharding operations")

then we can either:

  1. no-assign the flag, and completely ignore any values the user sets, e.g.
	_ = subFlags.String("sharding_column_name", "", "Specifies the column to use for sharding operations")

OR

  1. we can do what @systay did for planner-version / planner_version here

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
@arvind-murty
Copy link
Contributor Author

Hey @arvind-murty, I went ahead and created a branch that removes those fields from vtctld2, you can rebase off #10459 to get those changes onto your branch!

Thank you. Since your PR was merged, I just did a merge from main.

Copy link
Collaborator

@deepthi deepthi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look fine.
I'm in two minds about whether we need to add this to the summary doc. v2 resharding (not the same as vreplication v2) is old enough that probably no one cares.
We are announcing the deprecation of these fields for the CLI commands in v14.
@ajm188 @frouioui do you have any opinions on this?

@ajm188
Copy link
Contributor

ajm188 commented Jun 10, 2022

I'm in two minds about whether we need to add this to the summary doc. v2 resharding (not the same as vreplication v2) is old enough that probably no one cares.
We are announcing the deprecation of these fields for the CLI commands in v14.
@ajm188 @frouioui do you have any opinions on this?

It's been marked as "legacy" since at least v13, so I think it's fine to just delete. If we feel a little iffy about it, we can include a "minor deletions/deprecations" section or something like that towards the bottom of the doc.

@deepthi deepthi merged commit 23727bf into vitessio:main Jun 10, 2022
timvaillancourt pushed a commit to slackhq/vitess that referenced this pull request Aug 16, 2023
* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* generate vtadmin files

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type from vtadmin

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* Merge from main

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
timvaillancourt pushed a commit to slackhq/vitess that referenced this pull request Aug 16, 2023
* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* generate vtadmin files

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type from vtadmin

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* Merge from main

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
timvaillancourt added a commit to slackhq/vitess that referenced this pull request Aug 30, 2023
…test + compressors (#119)

* Delete all legacy sharding related code (vitessio#10278)

* Delete all legacy sharding related code

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Move used until functions from initialsharding to cluster

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove vtctl commands

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Kill vtworker and SetKeyspaceServedFrom cmd

Signed-off-by: Matt Lord <mattalord@gmail.com>

* WaitForDrain related stragglers

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Legacy local straggler workflow

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Get rid of SetKeyspaceShardingInfo & wait for drain stragglers

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove vtworker stragglers

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Update throttlerservice protobuf

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Rename test 24, add hashicorp vault test to it (now mysql_server_vault)

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove last mentiones of legacy sharding in vtctl

Signed-off-by: Matt Lord <mattalord@gmail.com>

* remove binlog_use_v3_resharding_mode

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Address review comments

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Address review comments

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Correct vtgate help output

Signed-off-by: Matt Lord <mattalord@gmail.com>

* go fmt

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Remove v2 resharding fields (vitessio#10409)

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* generate vtadmin files

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type from vtadmin

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* Merge from main

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* Fix bad merge conflict resolution

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Fix missing 'sharding' import

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Fix bad conflict resolution in go/test/endtoend/cellalias/cell_alias_test.go

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Backup/Restore: add support for external compressors and decompressors (vitessio#10558)

* change to support an external decompressor

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* add external compressor support + builtin additional compressors

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* wrap external compressor/decompressor

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* go mod tidy + comments

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* add copyright notices

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* add support for builtin engine

Signed-off-by: Renan Rangel <rrangel@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Adding test case for buckup compression

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fixing unit test and run mod tidy

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Removing unwanted unit tests

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Increase timeout of backup tests

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* fixing linter errors

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Change test logic to accomodate running selective tests

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* removing lint warning

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* fixing test failure

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Removing un-necessary test

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fixing code review feeback

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Change builtinEngine to consider 'auto' decompressor

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* fixing Upgrade/Downgrade test

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fix type & add summary under release notes

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fixing typos in summary

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fixing flag name typos

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

Co-authored-by: Renan Rangel <rrangel@slack-corp.com>
Co-authored-by: Renan Rangel <renan@slack-corp.com>

* Add MySQL 8 Support to Backup Tests (vitessio#10691)

* Add support for MySQL 8.0 in backup tests

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add 8.0 workflow

Signed-off-by: Matt Lord <mattalord@gmail.com>

* whitespace

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use vtctldclient SetKeyspaceDurabilityPolicy to manage semi-sync

This needed to be done after the shard was setup in order to satisfy
the semantic assumptions related to semi-sync in the tests.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove extraneous changes

Signed-off-by: Matt Lord <mattalord@gmail.com>

* We need lz4 for TestXtrabackupStreamWithlz4Compression

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Try using Percona Repo for MySQL 8 to align mysqld and xtrabackup versions

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Specify stream type everywhere

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove repeated server install

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Moar...

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Move vtctlbackup test to 8.0

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Rename vtbackup test and move to MySQL 8

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Split the xbstream tests so the workflow doesn't time out

Otherwise it was going over the 10min limit and getting
killed.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use MySQL 8 compat method for setting passwords

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Test increasing timeout at another level

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Don't use the init passwords file with 8.0

mysqlctl doesn't start...

This is likely due to the change in initialization behavior with
MySQL 8.0 as it goes though two phases and you can't simply
start up mysqld and pass it data, it has to initialize and restart
first.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Fix incorrect password update statement for vt_repl user

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Bump timeouts for 8.0 backup tests

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Bump it more :(

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Increase backup/restore timeout in backup_utils

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Apply new 8.0 template everywhere

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Fix bugs around how the compression flags were getting passed

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use 45m timeout for the workflow, 30m for the run.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* These changes were no longer needed so limiting diff.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Explicitly skip new linter check

Signed-off-by: Matt Lord <mattalord@gmail.com>

Co-authored-by: Rohit Nayak <rohit@planetscale.com>

* Fix test file merge issues

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Go 1.18.7 to fix error

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Fix lint

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Fix vttablet.txt

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Remove sharding_column_name and sharding_column_type from vtctld2 (vitessio#10459)

* Remove sharding_column_name and sharding_column_type

Signed-off-by: notfelineit <notfelineit@gmail.com>

* Run make web_build to compile production files

Signed-off-by: notfelineit <notfelineit@gmail.com>

* Remove vars from test

Signed-off-by: notfelineit <notfelineit@gmail.com>

* Remove sharding column type select box test from web test

Signed-off-by: notfelineit <notfelineit@gmail.com>

---------

Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
Co-authored-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: Arvind Murty <arvind.murty@gmail.com>
Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com>
Co-authored-by: Renan Rangel <rrangel@slack-corp.com>
Co-authored-by: Renan Rangel <renan@slack-corp.com>
Co-authored-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: Frances Thai <31225471+notfelineit@users.noreply.github.com>
timvaillancourt added a commit to slackhq/vitess that referenced this pull request Sep 1, 2023
…test + compressors (#119)

* Delete all legacy sharding related code (vitessio#10278)

* Delete all legacy sharding related code

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Move used until functions from initialsharding to cluster

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove vtctl commands

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Kill vtworker and SetKeyspaceServedFrom cmd

Signed-off-by: Matt Lord <mattalord@gmail.com>

* WaitForDrain related stragglers

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Legacy local straggler workflow

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Get rid of SetKeyspaceShardingInfo & wait for drain stragglers

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove vtworker stragglers

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Update throttlerservice protobuf

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Rename test 24, add hashicorp vault test to it (now mysql_server_vault)

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove last mentiones of legacy sharding in vtctl

Signed-off-by: Matt Lord <mattalord@gmail.com>

* remove binlog_use_v3_resharding_mode

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Address review comments

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Address review comments

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Correct vtgate help output

Signed-off-by: Matt Lord <mattalord@gmail.com>

* go fmt

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Remove v2 resharding fields (vitessio#10409)

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* generate vtadmin files

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* cleanup: remove sharding_column_name and sharding_column_type from vtadmin

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* Merge from main

Signed-off-by: Arvind Murty <arvind.murty@gmail.com>

* Fix bad merge conflict resolution

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Fix missing 'sharding' import

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Fix bad conflict resolution in go/test/endtoend/cellalias/cell_alias_test.go

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Backup/Restore: add support for external compressors and decompressors (vitessio#10558)

* change to support an external decompressor

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* add external compressor support + builtin additional compressors

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* wrap external compressor/decompressor

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* go mod tidy + comments

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* add copyright notices

Signed-off-by: Renan Rangel <renan@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* add support for builtin engine

Signed-off-by: Renan Rangel <rrangel@slack-corp.com>
Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Adding test case for buckup compression

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fixing unit test and run mod tidy

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Removing unwanted unit tests

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Increase timeout of backup tests

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* fixing linter errors

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Change test logic to accomodate running selective tests

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* removing lint warning

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* fixing test failure

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Removing un-necessary test

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fixing code review feeback

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Change builtinEngine to consider 'auto' decompressor

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* fixing Upgrade/Downgrade test

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fix type & add summary under release notes

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fixing typos in summary

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

* Fixing flag name typos

Signed-off-by: Rameez Sajwani <rameezwazirali@hotmail.com>

Co-authored-by: Renan Rangel <rrangel@slack-corp.com>
Co-authored-by: Renan Rangel <renan@slack-corp.com>

* Add MySQL 8 Support to Backup Tests (vitessio#10691)

* Add support for MySQL 8.0 in backup tests

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Add 8.0 workflow

Signed-off-by: Matt Lord <mattalord@gmail.com>

* whitespace

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use vtctldclient SetKeyspaceDurabilityPolicy to manage semi-sync

This needed to be done after the shard was setup in order to satisfy
the semantic assumptions related to semi-sync in the tests.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove extraneous changes

Signed-off-by: Matt Lord <mattalord@gmail.com>

* We need lz4 for TestXtrabackupStreamWithlz4Compression

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Try using Percona Repo for MySQL 8 to align mysqld and xtrabackup versions

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Specify stream type everywhere

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Remove repeated server install

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Moar...

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Move vtctlbackup test to 8.0

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Rename vtbackup test and move to MySQL 8

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Split the xbstream tests so the workflow doesn't time out

Otherwise it was going over the 10min limit and getting
killed.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use MySQL 8 compat method for setting passwords

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Test increasing timeout at another level

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Don't use the init passwords file with 8.0

mysqlctl doesn't start...

This is likely due to the change in initialization behavior with
MySQL 8.0 as it goes though two phases and you can't simply
start up mysqld and pass it data, it has to initialize and restart
first.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Fix incorrect password update statement for vt_repl user

Signed-off-by: Rohit Nayak <rohit@planetscale.com>

* Bump timeouts for 8.0 backup tests

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Bump it more :(

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Increase backup/restore timeout in backup_utils

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Apply new 8.0 template everywhere

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Fix bugs around how the compression flags were getting passed

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Use 45m timeout for the workflow, 30m for the run.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* These changes were no longer needed so limiting diff.

Signed-off-by: Matt Lord <mattalord@gmail.com>

* Explicitly skip new linter check

Signed-off-by: Matt Lord <mattalord@gmail.com>

Co-authored-by: Rohit Nayak <rohit@planetscale.com>

* Fix test file merge issues

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Go 1.18.7 to fix error

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Fix lint

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Fix vttablet.txt

Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>

* Remove sharding_column_name and sharding_column_type from vtctld2 (vitessio#10459)

* Remove sharding_column_name and sharding_column_type

Signed-off-by: notfelineit <notfelineit@gmail.com>

* Run make web_build to compile production files

Signed-off-by: notfelineit <notfelineit@gmail.com>

* Remove vars from test

Signed-off-by: notfelineit <notfelineit@gmail.com>

* Remove sharding column type select box test from web test

Signed-off-by: notfelineit <notfelineit@gmail.com>

---------

Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
Co-authored-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: Arvind Murty <arvind.murty@gmail.com>
Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com>
Co-authored-by: Renan Rangel <rrangel@slack-corp.com>
Co-authored-by: Renan Rangel <renan@slack-corp.com>
Co-authored-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: Frances Thai <31225471+notfelineit@users.noreply.github.com>
timvaillancourt added a commit to slackhq/vitess that referenced this pull request Sep 4, 2023
…test + compressors (#119) (#124)

* Delete all legacy sharding related code (vitessio#10278)

* Delete all legacy sharding related code



* Move used until functions from initialsharding to cluster



* Remove vtctl commands



* Kill vtworker and SetKeyspaceServedFrom cmd



* WaitForDrain related stragglers



* Legacy local straggler workflow



* Get rid of SetKeyspaceShardingInfo & wait for drain stragglers



* Remove vtworker stragglers



* Update throttlerservice protobuf



* Rename test 24, add hashicorp vault test to it (now mysql_server_vault)



* Remove last mentiones of legacy sharding in vtctl



* remove binlog_use_v3_resharding_mode



* Address review comments



* Address review comments



* Correct vtgate help output



* go fmt



* Remove v2 resharding fields (vitessio#10409)

* cleanup: remove sharding_column_name and sharding_column_type



* cleanup: remove sharding_column_name and sharding_column_type



* cleanup: remove sharding_column_name and sharding_column_type



* cleanup: remove sharding_column_name and sharding_column_type



* generate vtadmin files



* cleanup: remove sharding_column_name and sharding_column_type from vtadmin



* Merge from main



* Fix bad merge conflict resolution



* Fix missing 'sharding' import



* Fix bad conflict resolution in go/test/endtoend/cellalias/cell_alias_test.go



* Backup/Restore: add support for external compressors and decompressors (vitessio#10558)

* change to support an external decompressor




* add external compressor support + builtin additional compressors




* wrap external compressor/decompressor




* go mod tidy + comments




* add copyright notices




* add support for builtin engine




* Adding test case for buckup compression



* Fixing unit test and run mod tidy



* Removing unwanted unit tests



* Increase timeout of backup tests



* fixing linter errors



* Change test logic to accomodate running selective tests



* removing lint warning



* fixing test failure



* Removing un-necessary test



* Fixing code review feeback



* Change builtinEngine to consider 'auto' decompressor



* fixing Upgrade/Downgrade test



* Fix type & add summary under release notes



* Fixing typos in summary



* Fixing flag name typos






* Add MySQL 8 Support to Backup Tests (vitessio#10691)

* Add support for MySQL 8.0 in backup tests



* Add 8.0 workflow



* whitespace



* Use vtctldclient SetKeyspaceDurabilityPolicy to manage semi-sync

This needed to be done after the shard was setup in order to satisfy
the semantic assumptions related to semi-sync in the tests.



* Remove extraneous changes



* We need lz4 for TestXtrabackupStreamWithlz4Compression



* Try using Percona Repo for MySQL 8 to align mysqld and xtrabackup versions



* Specify stream type everywhere



* Remove repeated server install



* Moar...



* Move vtctlbackup test to 8.0



* Rename vtbackup test and move to MySQL 8



* Split the xbstream tests so the workflow doesn't time out

Otherwise it was going over the 10min limit and getting
killed.



* Use MySQL 8 compat method for setting passwords



* Test increasing timeout at another level



* Don't use the init passwords file with 8.0

mysqlctl doesn't start...

This is likely due to the change in initialization behavior with
MySQL 8.0 as it goes though two phases and you can't simply
start up mysqld and pass it data, it has to initialize and restart
first.



* Fix incorrect password update statement for vt_repl user



* Bump timeouts for 8.0 backup tests



* Bump it more :(



* Increase backup/restore timeout in backup_utils



* Apply new 8.0 template everywhere



* Fix bugs around how the compression flags were getting passed



* Use 45m timeout for the workflow, 30m for the run.



* These changes were no longer needed so limiting diff.



* Explicitly skip new linter check





* Fix test file merge issues



* Go 1.18.7 to fix error



* Fix lint



* Fix vttablet.txt



* Remove sharding_column_name and sharding_column_type from vtctld2 (vitessio#10459)

* Remove sharding_column_name and sharding_column_type



* Run make web_build to compile production files



* Remove vars from test



* Remove sharding column type select box test from web test



---------

Signed-off-by: Matt Lord <mattalord@gmail.com>
Signed-off-by: Tim Vaillancourt <tim@timvaillancourt.com>
Signed-off-by: Arvind Murty <arvind.murty@gmail.com>
Signed-off-by: notfelineit <notfelineit@gmail.com>
Co-authored-by: Matt Lord <mattalord@gmail.com>
Co-authored-by: Arvind Murty <arvind.murty@gmail.com>
Co-authored-by: Rameez Sajwani <rameezwazirali@hotmail.com>
Co-authored-by: Renan Rangel <rrangel@slack-corp.com>
Co-authored-by: Renan Rangel <renan@slack-corp.com>
Co-authored-by: Rohit Nayak <rohit@planetscale.com>
Co-authored-by: Frances Thai <31225471+notfelineit@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: General Changes throughout the code base Type: Internal Cleanup

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Task: delete unused sharding_column_name and sharding_column_type

4 participants