[vtgate planner] Routing & Merging refactor

[systay]

Description

This PR refactors how routing of queries is done during query planning.

Why?

The logic for which routes can be merged together is an important and complex part of the query planner.
Making the code easy to understand and talk about is critical to get this correct.

The old Route operator consisted of a set of fields:

type Route struct {
    Source              ops.Operator
    RouteOpCode         engine.Opcode
    Keyspace            *vindexes.Keyspace
    VindexPreds         []*VindexPlusPredicates
    Selected            *VindexOption
    SysTableTableSchema []evalengine.Expr
    SysTableTableName   map[string]evalengine.Expr
    SeenPredicates      []sqlparser.Expr
    TargetDestination   key.Destination
    Alternates          map[*vindexes.Keyspace]*Route
    MergedWith          []*Route
}

Of these, only Source, RouteOpCode and MergedWith are valid for all types of routes.
All other fields only make sense for some OpCodes that the route represents.

The fields VindexPreds and Selected only make sense for sharded tables, which are represented a lot of OpCodes, such as Scatter, EqualUnique, etc.
SysTableTableSchema, SysTableTableName are only used for information_schema tables (OpCode DBA).

In a lot of places, we had to use a switch statement on the OpCode to handle things differently depending on the type of Route we were dealing with.

The Change

To me, this screamed for an interface and multiple different implementation of this interface, depending on which type of route we have.
The new operator now contains:

Route struct {
    Source     ops.Operator
    MergedWith []*Route
    Routing    Routing
}

The Routing interface is then used for picking the best plan per table in the query, and then the merging of multiple Routes into as few as possible.

While doing this refactoring, I tried to keep the tests intact and only change the code behind. For the few exceptions to this rule, I have added comments in this PR explaining why the change was introduced.

Checklist

• "Backport to:" labels have been added if this change should be back-ported
• Tests were added or are not required
• Documentation was added or is not required

[vitess-bot]

Review Checklist

Hello reviewers! 👋 Please follow this checklist when reviewing this Pull Request.

General

• Ensure that the Pull Request has a descriptive title.
• If this is a change that users need to know about, please apply the release notes (needs details) label so that merging is blocked unless the summary release notes document is included.
• If a test is added or modified, there should be a documentation on top of the test to explain what the expected behavior is what the test does.

If a new flag is being introduced:

• Is it really necessary to add this flag?
• Flag names should be clear and intuitive (as far as possible)
• Help text should be descriptive.
• Flag names should use dashes (-) as word separators rather than underscores (_).

If a workflow is added or modified:

• Each item in Jobs should be named in order to mark it as required.
• If the workflow should be required, the maintainer team should be notified.

Bug fixes

• There should be at least one unit or end-to-end test.
• The Pull Request description should include a link to an issue that describes the bug.

Non-trivial changes

• There should be some code comments as to why things are implemented the way they are.

New/Existing features

• Should be documented, either by modifying the existing documentation or creating new documentation.
• New features should have a link to a feature request issue or an RFC that documents the use cases, corner cases and test cases.

Backward compatibility

• Protobuf changes should be wire-compatible.
• Changes to _vt tables and RPCs need to be backward compatible.
• vtctl command output order should be stable and awk-able.
• RPC changes should be compatible with vitess-operator
• If a flag is removed, then it should also be removed from VTop, if used there.

[systay]

We are now using the evalengine to check if we can at plan-time evaluate expression. If we can, and if the result is false, we can use the None opcode which is very cheap.

[systay]

The old query was wrong - we should never use the given schema name. Instead we have to replace the literal value with the argument :__vtschemaname which is then filled in by the vttablet with the name of the underlying MySQL database.

[systay]

The old plan was wrong. Given WHERE kcu.table_schema = ? AND rc.constraint_schema = ?, we won't know until runtime if the user wants to look at information from the same or different keyspaces/schemas, and so merging these routes into a single one is invalid.

[systay]

Since we have merged the two routes, there is no need to specify the table_schema schema twice.

[systay]

Here we know that the two schemas being searched for are different - WHERE tc.table_schema = 'table_schema' AND ... cc.constraint_schema = 'constraint_schema'. Merging these two was wrong.

[systay]

ambiguous_ref_with_source exists both as an unsharded table in the main keyspace, and as a reference table in the user keyspace. The latter is a copy of the unsharded table spread out to all shards so that all joins can be local.

During route planning we have decided that we want to send the query to the unsharded main keyspace. The OpCode is more accurate if it's Unsharded for this route.

[frouioui]

This is great, it's an important change that makes things easier for all of us and that fills some gaps we had, thank you @systay 🙏🏻

I left a few questions, comments and nits after this first pass

[harshit-gangal]

now that we do not merge if SysTableTableSchema is different, should SysTableTableSchema be an Expr than slice of Expr? Similarly, Do we need SysTableTableName as a map?

[systay]

valid points. wdyt about doing these fixes in a separate PR? this one has grown enough for now :)

[harshit-gangal]

We should add a task for it

[frouioui]

Looks good to me!