Skip to content

Added ValidateVSchema#8012

Merged
ajm188 merged 1 commit intovitessio:masterfrom
tinyspeck:validate-keyspace-vschema-checks
May 11, 2021
Merged

Added ValidateVSchema#8012
ajm188 merged 1 commit intovitessio:masterfrom
tinyspeck:validate-keyspace-vschema-checks

Conversation

@makinje16
Copy link
Copy Markdown
Contributor

@makinje16 makinje16 commented Apr 30, 2021

Signed-off-by: Malcolm Akinje makinje@slack-corp.com

Description

This PR adds in ValidateVSchema. ValidateSchemaShard now uses this method as a helper when includeVSchema is provided to it. This also gives the oppourtunity to validate that a shards schema is equivalent to the vschema in other areas besides the aforementioned method like when we create a resharding workflow.

Related Issue(s)

Checklist

  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

@makinje16 makinje16 marked this pull request as ready for review April 30, 2021 20:59
Copy link
Copy Markdown
Contributor

@ajm188 ajm188 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there, I think. I think the current refactor makes the ValidateSchemaShard a little overloaded, and pulling out the vschema validation to its own function will make this a bit clearer.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a flag to this command (and probably ValidateSchemaShard, sorry for not pointing that out earlier) to be able to pass true through here.

@makinje16 makinje16 force-pushed the validate-keyspace-vschema-checks branch from 499c991 to 5d90c86 Compare May 4, 2021 20:20
@makinje16 makinje16 changed the title Added includeVSchema to ValidateSchemaKeyspace Added ValidateVSchema and ValidateVSchemaKeyspace May 4, 2021
@makinje16 makinje16 requested a review from ajm188 May 4, 2021 21:25
Copy link
Copy Markdown
Contributor

@ajm188 ajm188 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything I have is pretty minor changes; I think Rafa was saying he had other thoughts so we should wait for him as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should be false?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed 👍

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You shouldn't need to pass a reference to the recorder here; it's in scope in the closure of the goroutine (similar to how you're using the wait group).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually got linting errors when I did not pass in the recorder or shard which is why I passed them in

Linting go/vt/wrangler
go/vt/wrangler/schema.go:289:45: loopclosure: loop variable shard captured by func literal (govet)
			si, err := wr.ts.GetShard(ctx, keyspace, shard)
			                                         ^
go/vt/wrangler/schema.go:291:83: loopclosure: loop variable shard captured by func literal (govet)
				shardFailures.RecordError(fmt.Errorf("GetShard(%v, %v) failed: %v", keyspace, shard, err))
				                                                                              ^
go/vt/wrangler/schema.go:297:45: loopclosure: loop variable shard captured by func literal (govet)
					excludeTables, includeViews, keyspace, shard, err,
					                                       ^

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those lint errors are complaining about the shard variable specifically (which is a correct error). I'm talking about just the shardFailures variable which isn't part of the range loop.

To illustrate what the linter is warning about, check out the difference between these two loops: https://play.golang.org/p/eQtSzSI60lR

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is unused?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is unused currently but thought that this functionality may be needed at some point. Instead of requiring the caller to pass in all shard names to ValidateVSchema we can use this. In the two instances in which I've seen us validate the vschema we also have had the shards we are trying to validate so I understand if we should remove this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think if we don't need it now (and we don't expect to immediately need it in the short-term), then we should hold off. We can always add it back later!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unused?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

@makinje16 makinje16 force-pushed the validate-keyspace-vschema-checks branch 2 times, most recently from 59e4a05 to 3c847b2 Compare May 7, 2021 16:18
@makinje16 makinje16 changed the title Added ValidateVSchema and ValidateVSchemaKeyspace Added ValidateVSchema May 7, 2021
Signed-off-by: Malcolm Akinje <makinje@slack-corp.com>
@makinje16 makinje16 force-pushed the validate-keyspace-vschema-checks branch from 3c847b2 to 3479947 Compare May 10, 2021 14:56
@makinje16 makinje16 requested a review from ajm188 May 10, 2021 16:08
Copy link
Copy Markdown
Member

@rafael rafael left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for addressing all the comments. This looks good to me.

Copy link
Copy Markdown
Contributor

@ajm188 ajm188 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me! thanks Malcolm ❤️

@ajm188 ajm188 merged commit 7a7a7c7 into vitessio:master May 11, 2021
@ajm188 ajm188 deleted the validate-keyspace-vschema-checks branch May 11, 2021 15:35
@askdba askdba added Component: Cluster management Type: Enhancement Logical improvement (somewhere between a bug and feature) labels Jun 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Component: Cluster management Type: Enhancement Logical improvement (somewhere between a bug and feature)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants