[vtadmin-api] Add static file service discovery implementation

[doeg]

Signed-off-by: Sara Bee 855595+doeg@users.noreply.github.com

Description

Two changes in this PR:

• Adds a service discovery implementation using static files
• Exposes the vtgate gRPC port :15991 in ./docker/local/run.sh so that vtadmin (running outside of Docker) can issue requests

The best part is that now we can run vtadmin-api and vtadmin-web locally, using a local Vitess (in Docker or otherwise). 🎉 So we can add data fetching (and real data) to vtadmin-web 😈 finally the fun begins.

We can enhance this later on by watching the cluster config file(s) for changes so updates don't require a restart of vtadmin-api.

Many thanks as always to @ajm188 for answering my eight million golang questions! 💛

Running it locally

First, you'll want to run Vitess itself locally. 😸 I always run Vitess using Docker but this works fine with uncontainerized local Vitess too.

(@shlomi-noach, thank you again for the fix in #7213!)

# Note that you'll want to use Shlomi's fix in PR #7213 for this step...
make docker_local

# ... and then switch back to this branch to run it, so that we expose the gRPC port 15991
./docker/local/run.sh

Second, you'll need to create an empty vtgate credentials file to avoid the gRPC dialer bug mentioned in #7187. Location and filename don't matter since you'll be passing this in as a flag; I put mine at /Users/sarabee/id1-grpc_vtgate_credentials.json:

{
	"Username": "",
	"Password": ""
}

Third, you'll want to create a cluster configuration file. This is the "static file" part of "static file service discovery". Assuming you're using the standard local Vitess set-up, you can steal this file as-is. (It comes with an entry for a bogus vtgate host to demonstrate that tagging works.) Again, filename and location don't matter since we'll be passing in the path as a flag; I put mine at /Users/sarabee/vtadmin-cluster1.json.

{
	"vtgates": [
		{
			"host": {
				"hostname": "127.0.0.1:15991"
			},
			"tags": ["pool:pool1", "cell:zone1", "extra:tag"]
		},
		{
			"host": {
				"hostname": "127.0.0.1:15992"
			},
			"tags": ["dead-dove-do-not-eat"]
		}
	]
}

Fourth, build this branch if you haven't already:

make build

Fifth, start up vtadmin-api but make sure to update the filepaths for the vtgate creds file and static service discovery file you created above!

./bin/vtadmin \
    --addr ":15999" \
    --cluster-defaults "vtsql-credentials-path-tmpl=/Users/sarabee/id1-grpc_vtgate_credentials.json" \
    --cluster "name=cluster1,id=id1,discovery=staticFile,discovery-staticFile-path=/Users/sarabee/vtadmin-cluster1.json,vtsql-discovery-tags=cell:zone1" 

Sixth and finally, we can curl vtadmin-api and get a valid response! 🎉

$ curl 127.0.0.1:15999/api/tablets

{"result":{"tablets":[{"cluster":{"id":"id1","name":"cluster1"},"tablet":{"alias":{"cell":"zone1","uid":100},"hostname":"2780467e90c9","keyspace":"commerce","shard":"0","type":1,"master_term_start_time":{"seconds":1609176750}},"state":1},{"cluster":{"id":"id1","name":"cluster1"},"tablet":{"alias":{"cell":"zone1","uid":101},"hostname":"2780467e90c9","keyspace":"commerce","shard":"0","type":2},"state":1},{"cluster":{"id":"id1","name":"cluster1"},"tablet":{"alias":{"cell":"zone1","uid":102},"hostname":"2780467e90c9","keyspace":"commerce","shard":"0","type":3},"state":1}]},"ok":true}

Related Issue(s)

• Initial vtadmin-api, clusters, and service discovery #7187 Initial vtadmin-api PR which contains the Consul service discovery implementation.

Checklist

• Should this PR be backported?
• Tests were added or are not required
• Documentation was added or is not required

Deployment Notes

N/A since vtadmin isn't in the deployment path :D

Impacted Areas in Vitess

Components that this PR will affect:

• Query Serving
• VReplication
• Cluster Management
• Build

[dkhenry]

Looks good to me. If you do have two two requests I would make are

1. Add in a full example json as a comment in the main implementation file.
2. Add in malformed json to one of the tests.

[doeg]

Looks good to me. If you do have two two requests I would make are

1. Add in a full example json as a comment in the main implementation file.
2. Add in malformed json to one of the tests.

@dkhenry Oh both really good suggestions! Thank you, I will do this. :D

[rohit-nayak-ps]

lgtm