Auto-registering Resources and System Clock issues.

[gileswr]

We are currently evaluating Teleport at our company. We have a couple of stumbling blocks we're looking to overcome that i'm sure others must have come across.

Firstly, we have several thousand resources we would need to regsiter with a cluster. Doing this manually one-by-one would not be feasible. They are Debian-based hosts configured using Ansible. Are Join Tokens the way forward for this?

Secondly - Clock synchronization: the certificates require time synchronization and on these hosts, the clock can drift. This is normally resolved using NTP, but if there is a connectivity issue, this could fail. If the batteries fail we could end up back in the 70s. Any failsafe way of accessing remote hosts in this event?

Thanks!

[webvictim]

1. It sounds like you would be best off creating an Ansible playbook to install Teleport, which takes a join token as input. You can use a regular alphanumeric join token valid for 1 hour (from tctl tokens add --type=node --ttl=1h) as many times as you like within that hour, so this would work to install Teleport on several thousand hosts at once.

2. Teleport does require the clock to be in sync for access to hosts to work, as it checks the issue/expiry times of the certificate issued to the user and will refuse a connection if the certificate appears to have been issued too far in the past/future. All the break glass strategies we have also use Teleport-issued certificates for security, so if the clock is out of sync on the host then these will also fail.

In this unique situation, you may be better off using a regular SSH key-based strategy with sshd for these hosts - run SSH with key-based authentication only on a high port, keep the private key offline in a secure location, and if the time drift won't automatically fix itself, get the private key and log into the host to fix it.

[gileswr]

Thanks for this - yes I had wondered if it might be the case that we have to run regular key-based SSH alongside teleport. The concern with this would be if an analyst has the key then what is stopping them making a copy and just using that in future. We could lock this ssh access down to a teleport host where the session could be monitored, but that’s still a bit of a weakness. I guess we could put something in place to monitor the usage of they key…

[webvictim]

Yes. I'd advise restricting the use of such a key to one specific breakglass account which rings every SIEM/monitoring alarm bell possible when used, so it's only ever feasible during emergencies. Also rotate the key after an outage so it's not possible to keep a copy and use it again in future. Definitely arduous, but hopefully very very rarely used.

I'm sure you can understand why my ultimate recommendation would probably be to run some kind of local network NTP source for the machines so they don't require internet access to maintain time sync!