Add agentless installer in the teleport discovery service

[lxea]

This is partially work in progress as it requires support for agentless inventory management in order to have ssh nodes show in the teleport ui/tsh ls output

Agentless discovery allows the discovery service to update the OpenSSH config in EC2 instances in order to add the teleport CA and enable certificate based authentication.

The following is executed on discovered nodes

  sudo /usr/bin/teleport join \
	  --openssh-config=/etc/ssh/sshd_config \
	  --join-method=iam \
	  --token="$1" \
	  --proxy-server="{{ .PublicProxyAddr }}" \
	  --additional-principals="$PUBLIC_IP" \
	  --restart-sshd

This will update the openssh config with the following configuration options

### Section created by 'teleport  join'
TrustedUserCaKeys /etc/ssh/teleport_user_ca.pub
HostKey /etc/ssh/teleport
HostCertificate /etc/ssh/teleport-cert.pub
### Section end

It will also write the certs and keys.

Agentless discovery can be enabled with the following config

discovery_service:
  enabled: yes
  aws:
    - types: ["ec2"]
      regions: ["eu-central-1"]
      install:
        install_teleport: false # enables agentless installation

[greedy52]

what happens when the instance has no public IP?

[lxea]

I'm not sure, I'm also not sure what the most appropriate behavior here is in the first place, should it also include a private ip in the principals maybe?

[lxea]

Updated this to also include the local-ipv4

[greedy52]

interesting. first time seeing authclient. we have so many different ways connecting.

[jakule]

If this write fails for any reason, we will leave the teleportKey on the disk. The same apply to teleportUserCA (then teleportCert and teleportKey will be left). Should we remove it on the write error?

[lxea]

I'm not sure which would be preferred, I'm inclined to leave them there, maybe a log message that there may be files left behind if writekeys has an error could be good?

[jakule]

@r0mant Any thoughts? I think that if the installation fails, we should leave the machine in the same state as if the installation would never happen. Otherwise, we may encounter weird errors on the second attempt.

[r0mant]

I agree, we should clean stuff up.

[r0mant]

But the code should tolerate files potentially being already present there just in case we failed to clean them up or something.

[jakule]

tolerate files potentially being already

I think this is more complicated in practice. First of all, we already recommend installing our certificates in /etc/ssh for proxy recording https://goteleport.com/docs/server-access/guides/recording-proxy-mode/
If we decide to "ignore already existing files", we may override some existing files not created by Teleport. TBH now when I think about it, I don't see why we would keep our certificates or any other files outside of /etc/teleport. I think that the only file that we need to modify outside of that directory is sshd config.
This is what I would do:

1. Create /etc/teleport if doesn't exist
2. Create /etc/teleport/agentless directory if doesn't exist. If the directory already exists, remove it (that should cover "failed previous installations" case).
3. Write all certs and keys inside /etc/teleport/agentless.
4. Create a backup of the sshd config (/etc/ssh/sshd_config.teleport maybe?).
5. Update the original sshd config and test it.
6. Restart sshd daemon.

In case of any failures, we would need to just remove /etc/teleport/agentless. This directory should also be added to our RPM/DEB packages to be removed on "uninstall" comments. In case of sshd failures, we should still be able to revert all changes as we will have the backed-up config.

That being said, this process (current and described here) will fail if a machine has SELinux enabled as Teleport process even running as a root won't have access to modify sshd config. I don't think that SELinux support should be included in the first implementation anyways. I just want to mention that we should be prepared to see errors related to it.

[lxea]

1. Create `/etc/teleport` if doesn't exist

2. Create `/etc/teleport/agentless` directory if doesn't exist. If the directory already exists, remove it (that should cover "failed previous installations" case).

3. Write all certs and keys inside `/etc/teleport/agentless`.

4. Create a backup of the sshd config (`/etc/ssh/sshd_config.teleport` maybe?).

5. Update the original sshd config and test it.

6. Restart sshd daemon.

Updated to do the above

In case of any failures, we would need to just remove /etc/teleport/agentless. This directory should also be added to our RPM/DEB packages to be removed on "uninstall" comments. In case of sshd failures, we should still be able to revert all changes as we will have the backed-up config.

Where do I need to update the RPM/DEB packages for this? Is it in teleport.git?

[jakule]

No idea. We're using https://fpm.readthedocs.io/en/v1.15.0/ so this is probably a good place to start.
TBH I'd just create a TODO comment and address that later. I don't think we have any custom script already added to DEB/RPM and adding the first one can be a pain (we could introduce some potential side effects).

@tcsc Correct me on the custom RPM/DEB scripts if I'm wrong.

[jakule]

I know this is not the last part, so I'm fine approving it, but can we add more tests to this functionality later?