MiniVitess: self contained vitess cluster, automatically resolving and self-configuring to meet external MySQL cluster

[shlomi-noach]

This PR offers MiniVitess. It's a single container that runs a complete vitess cluster, and which probes and analyzes an external, existing MySQL replication topology, and self configures to meet said cluster.

Breakdown of what MiniVitess does:

• Receives from the user hostname[:port] for some MySQL server in a topology
• Receives schema name
• Receives MySQL user+password
  Right now this user+password is used by both vitess as an app, as well as orchestrator as a discovery service
• Sets up etc
• Sets up vtctld
• Runs orchestrator to probe given MySQL instance
• orchestrator probes and maps entire replication topology
• For each server in th etopology, spin up a vttablet
• Self identify the master of the topology, externally reparent
• Spin up vtgate
• expose ports (ie vtgate, vtctld, vttablets web API are visible to the outside)

At this point you have a (single shard) vitess cluster you can use to query your existing MySQL replication topology.

See the below bootstrap output to get a bit more sense of what it looks like:

$ make docker_mini

$ ./docker/mini/run.sh -s "172.17.0.1:10111" -d "test" -u "ci" -p "ci"
=============================================

|  \/  (_)_ __ (_) \   / (_) |_ ___  ___ ___ 
| |\/| | | '_ \| |\ \ / /| | __/ _ \/ __/ __|
| |  | | | | | | | \ V / | | ||  __/\__ \__ \
|_|  |_|_|_| |_|_|  \_/  |_|\__\___||___/___/

=============================================
MiniVitess is starting up...
- Given MySQL topology server: 172.17.0.1:10111
- Given MySQL schema: test

# orchestrator
- Configuring orchestrator with given topology server and credentials...
- Starting orchestrator... Logfile is /vt/vtdataroot/tmp/orchestrator.out
- orchestrator will meanwhile probe and map your MySQL replication topology

# etcd
add /vitess/global
add /vitess/zone1
add zone1 CellInfo
ERROR: logging before flag.Parse: E0625 07:31:09.411292      63 syslogger.go:122] can't connect to syslog
W0625 07:31:09.421074      63 vtctl.go:89] cannot connect to syslog: Unix syslog delivery error
etcd start done...

# vtctld
- Starting vtctld...
+ started

# Waiting for topology to be detected and analyzed
...............

# orchestrator has detected the following MySQL servers:
172.17.0.1:10111
172.17.0.1:10112
172.17.0.1:10113
172.17.0.1:10114

# Bootstrapping vttablets for the above instances...
> Starting vttablet for server 172.17.0.1:10111
  - Tablet alias is zone1-0000000100
  - Tablet listens on http://127.0.0.1:15100
  + vttablet started
  > Setting this tablet as master
  + done
> Starting vttablet for server 172.17.0.1:10112
  - Tablet alias is zone1-0000000101
  - Tablet listens on http://127.0.0.1:15101
  + vttablet started
> Starting vttablet for server 172.17.0.1:10113
  - Tablet alias is zone1-0000000102
  - Tablet listens on http://127.0.0.1:15102
  + vttablet started
> Starting vttablet for server 172.17.0.1:10114
  - Tablet alias is zone1-0000000103
  - Tablet listens on http://127.0.0.1:15103
  + vttablet started

# vtgate
Waiting for vtgate to be up...
vtgate is up!
Access vtgate at http://127.0.0.1:15001/debug/status

# Setup complete

# Your topology is:
172.17.0.1:10111   0s ok 5.7.26-log rw ROW >>,GTID
+ 172.17.0.1:10112 0s ok 5.7.26-log ro ROW >>,GTID
+ 172.17.0.1:10113 0s ok 5.7.26-log ro ROW >>,GTID
+ 172.17.0.1:10114 0s ok 5.7.26-log ro ROW >>,GTID

# vtgate is listening for MySQL connection on port 15306

# On this container, try running any of the following commands:
  mysql test -e 'show tables'
  mysql test@master -e 'select @@hostname, @@port'
  mysql test@replica -e 'select @@hostname, @@port'

# Outside this container, try running any of the following commands:
  mysql -h sn-carbon --port=15306 test -e 'show tables'
  mysql -h sn-carbon --port=15306 test@master -e 'select @@hostname, @@port'
  mysql -h sn-carbon --port=15306 test@replica -e 'select @@hostname, @@port'

# Ready

The objective of this work is to (hopefully significantly) reduce the initial toll of experimenting with vitess. This setup allows for a relatively simple initial setup, so that one can see how vitess behaves, can see if their queries are supported, etc.

[shlomi-noach]

things to iterate on:

• different credentials for vitess app and for orchestrator app
• orchestrator discovery methods. Is it @@hostname? @@report_host? Everyone has it different.

[saifalharthi]

I mentioned VTCompose to @shlomi-noach in Slack. I thought they would do something similar (in some respects they do) but there are some differences. https://github.com/vitessio/vitess/tree/master/examples/compose

[shlomi-noach]

@saifalharthi thank you! Yeah, I was actually unaware of https://github.com/vitessio/vitess/tree/master/examples/compose and there seems to be quite an overlap. Will look more deeply into it.

If this PR turns to be completely redundant, it's still a day-and-a-half well spent on learning 🤷‍♂️

[derekperkins]

This also has some resemblance to

• https://github.com/vitessio/vitess/tree/master/go/cmd/vttestserver
• https://github.com/vitessio/vitess/tree/master/go/cmd/vtcombo

There's clearly a use case for this, whether that's experimenting with Vitess, using it locally to develop Vitess, or in a testing/CI environment as a dependency. There's a ton of overlap and we haven't been good about communicating the options. I think we should figure out what we want and consolidate all the functionality into a single binary/Docker image.

This adds a lot of cool discoverability to the existing options, and I think minivitess is a better name than either vttestserver or vtcombo, especially given that most people will be familiar with minikube.

[shlomi-noach]

There's a ton of overlap and we haven't been good about communicating the options.

To be fair I haven't done a deep investigation; I was for a reason under the impression that such setup did not exist.

[sougou]

I'd say vtcombo and vttestserver are overlapped. But this is fundamentally different because it brings up the vitess binaries themselves. It's probably closer to compose or minikube, but hopefully more lightweight.