Skip to content

Partial scatter with warnings#4293

Merged
sougou merged 9 commits intovitessio:masterfrom
tinyspeck:partial-scatter-with-warnings
Oct 29, 2018
Merged

Partial scatter with warnings#4293
sougou merged 9 commits intovitessio:masterfrom
tinyspeck:partial-scatter-with-warnings

Conversation

@demmer
Copy link
Copy Markdown
Member

@demmer demmer commented Oct 20, 2018

Description

Final stage of the SCATTER_ERRORS_AS_WARNINGS query directive support. (Part 4 of #4279 )

Details

This last step ties the three previous changes together.

  • When the partial scatter directive is set, the engine records any errors in the vtgate session as warnings, using a new RecordWarnings() method on the vcursor.
  • The count of these warnings are returned over the mysql protocol.
  • VtGate clears warnings from the session before each execution except for SHOW statements so that SHOW WARNINGS still works.
  • Add warning support to the python client and end to end tests that use this.

This includes adding a new RecordWarnings() method to the vcursor
interface so that primitives can store warnings in the session.

Also add support to the route primitive to record warnings when the
SCATTER_ERRORS_AS_WARNINGS query directive is set.

Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
To support the semantics of mysql warnings, clear them for all
"non-diagnostic" statements, which in this case can be reasoned
to be every statement other than SHOW.

Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Remove the hard-coded vitess-specific error code for a shard error
and instead extract the error code from the underlying error object
like we do for regular mysql protocol errors.

Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
@demmer demmer force-pushed the partial-scatter-with-warnings branch from 7f2cdd4 to c1a5a3c Compare October 28, 2018 17:33
@demmer
Copy link
Copy Markdown
Member Author

demmer commented Oct 28, 2018

@sougou Now that the other three PRs are merged, I rebased this on the latest master and updated the description. This is now ready for review.

@sougou
Copy link
Copy Markdown
Contributor

sougou commented Oct 28, 2018

Change is good. Tests need fixing.

Need to call NewSQLErrorFromError to parse the errno out of the
message since it isn't necessarily a SQLError.

Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
@demmer demmer force-pushed the partial-scatter-with-warnings branch from 882c12f to 9ca51ff Compare October 29, 2018 03:28
@demmer
Copy link
Copy Markdown
Member Author

demmer commented Oct 29, 2018

@sougou Updated with fixes for the end to end tests, which incidentally caught an oversight since I wasn't properly parsing into a SQLError to get the right mysql error code.

Copy link
Copy Markdown
Contributor

@sougou sougou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found one potential race (that I missed before). Looks good otherwise.

// for all statements _except_ SHOW, so that SHOW WARNINGS
// can actually return them.
if stmtType != sqlparser.StmtShow {
safeSession.Warnings = nil
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just occurred to me that this assignment can race with itself because of recursive calls through engine and vindex. It will be safer to mutex this.

Copy link
Copy Markdown
Member Author

@demmer demmer Oct 29, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.

But it’s not simply solved with a mutex. It also seems like the recursive vindex execute could clear warnings from the current session.

In the current use case I don’t think this is a problem because warnings get applied at the end of the route phase which comes after the vindex statements.

But this also means the partial scatter option doesn’t get propagated to the vindex statements themselves which it probably should.

That’s a long way of saying that this feature is still not totally defined for lookup-dependent plans.

IMO we should add the mutex to be race free and merge as is. Then we can fix this later.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

Signed-off-by: Michael Demmer <mdemmer@slack-corp.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants