Add database access for GCP Spanner

[GavinFrazar]

Changelog: Added support for GCP Spanner to Teleport Database Service.

This PR adds support for GCP Spanner to database access.

Implements https://github.com/gravitational/customer-sensitive-requests/issues/170

IAM Setup

1. You need a GCP service account with permissions to impersonate other identities. This can be an identity with the predefined role roles/iam.serviceAccountTokenCreator bound to it. It just needs iam.serviceAccounts.getAccessToken permission though.
2. Then you need another service account that has permissions to do things in GCP Spanner.
   There are some predefined roles you can bind to your service account like roles/spanner.databaseAdmin or roles/spanner.databaseUser.
   The latter should be sufficient to do whatever you'd like to test manually.
   See: https://cloud.google.com/spanner/docs/iam#roles

If it helps, this is the terraform module I made to setup a service account for teleport (the controller) and 3 service accounts that the controller can impersonate to access Spanner: https://github.com/GavinFrazar/infra/blob/master/terraform/modules/gcp-iam-spanner/main.tf

Teleport DB Setup

Create a database with the "spanner" protocol and the gcp project/instance ID spec populated, for example:

kind: db
version: v3
metadata:
  name: "teleport-spanner"
  description: "gcp spanner db"
  labels:
    env: "dev"
spec:
  protocol: "spanner"
 # this will default to google api endpoint "spanner.googleapis.com:443".
 # it's required to be that endpoint too.
  uri: ""
  gcp:
    project_id: "example-proj123"
    instance_id: "example-spanner"

Access

A spanner instance can contain multiple databases.
Teleport will enforce db_names for these databases for each RPC.

The --db-user you use corresponds to the name of a service account you wish to access the database as.
Note that this does not include the full "name@.iam.gservice.account.com", it's only the name, e.g.
if the service account is gavin-spanner@teleport-dev-123456.iam.gserviceaccount.com then I would specify --db-user=gavin-spanner

CLI

There's a cli I found that works pretty nicely, and you can install it with go install github.com/cloudspannerecosystem/spanner-cli@latest
Then just tsh db connect ... and it should work too. It also uses a local proxy tunnel under the hood.

GUI

Start a local proxy tunnel: tsh proxy db --tunnel --port 8080 --db-user=<user> --db-name=example-db teleport-spanner

Configure your client to connect to the local proxy, and ensure it also has the right project, instance, and database id, for example you can use this jdbc string in DataGrip: jdbc:cloudspanner://localhost:8080/projects/example-proj123/instances/example-spanner/databases/example-db
It follows the template "projects/<project>/instances/<instance>/databases/<database>"

You also need to configure your client to connect to the local proxy tunnel without tls and without GCP credentials. DataGrip and even the driver in go do not allow you to use any credentials without TLS, and DataGrip also will not use any client certs even if you use the custom ssl settings, unfortunately.
So tsh proxy db --tunnel is pretty much the only way to go here.

TODO

I will write a full docs guide in a follow up PR.

[public-teleport-github-review-bot]

@GavinFrazar - this PR will require admin approval to merge due to its size. Consider breaking it up into a series smaller changes.

[Tener]

A few comments from my first round. It was a lot to take in at once, is there a meaningful way to split this PR into smaller chunks?

[smallinsky]

First quick pass.

[GavinFrazar]

A few comments from my first round. It was a lot to take in at once, is there a meaningful way to split this PR into smaller chunks?

I will split the PR into:

1. configuration and client side changes (new db protocol, db object validation, tsh changes)
2. engine
3. web ui

we've already got some good discussion focused mostly on the engine. Let's treat this PR as just engine review. I'll change its merge base to get all the stuff from PR 1 out of the diff.

@Tener @smallinsky @greedy52

Edit: actually I wonder if that will be easier to review or just more tedious and difficult for reviewers to see the full picture of these changes. I kind of like that you can try out the full implementation with tsh and audit events in the web ui currently.
If you guys do want it split up, then please let me know whether the above outline sounds good.

[Tener]

Edit: actually I wonder if that will be easier to review or just more tedious and difficult for reviewers to see the full picture of these changes. I kind of like that you can try out the full implementation with tsh and audit events in the web ui currently.
If you guys do want it split up, then please let me know whether the above outline sounds good.

I'm personally in favour of splitting up. It is true that getting the full picture is harder once this is split, but I find it easier to provide quality feedback with these smaller PRs.

Managing a set of PRs is a bit of extra work too, especially if the chain is long, but I believe it leads to superior quality in the long run.

[greedy52]

This is amazing! Good job.
First quick pass. Will test out next round.
I think splitting would certainly make review easier and smaller PRs can go in a lot quicker. That said I don't mind reviewing PR of this size either.

[GavinFrazar]

I pulled out the tsh, lib/client, teleterm and webui stuff from this PR to make it smaller

[Tener]

That said, do we count common.GetMessagesFromServerMetric or common.GetMessagesFromClientMetric in spanner?

This should be easy to add via interceptor.

[GavinFrazar]

That said, do we count common.GetMessagesFromServerMetric or common.GetMessagesFromClientMetric in spanner?

This should be easy to add via interceptor.

I've added these metrics in a grpc.StatsHandler in spanner/grpcserver.go

@smallinsky friendly ping - this needs group 1

[zmb3]

/excludeflake *

[r0mant]

[public-teleport-github-review-bot]

@GavinFrazar See the table below for backport results.

Branch	Result
branch/v15	Failed