Teleport v3.0 introduces Kubernetes support
gravitational.com
gravitational.com
Dropping out of or below the cluster layer inherently means there is something in K8s that is missing. But no matter what is added to its APIs, Kubernetes will experience a long tail of weird corner cases that just can’t be covered. Reportedly the same is true with Borg jobs internally within Google.
Teleport's K8s protocol implementation mates its certificate-based SSH auth with Kubernetes certificate signing request API, plus Teleport acts as a full recording MITM for all K8s apiserver requests. How exactly end-users will leverage this remains to be seen, but we're super excited to see where the community takes this (and related tricky K8s vs provider IAM issues) as it continues to expand.
Full disclosure -- I work at Gravitational, worked closely with Sasha on this feature[1], and am a fan of Kelsey Hightower's live demo keynotes[2] where he frequently quips about the inherent struggle between procedural Dev+Ops encountering declarative APIs such as K8s.
[1] https://github.com/gravitational/teleport/issues/1986 [2] https://www.youtube.com/watch?v=07jq-5VbBVQ
Furthermore, one could declare (a bit like "live" coding) a one-liner (say in java) that would get called if that lines passes - so you can log something in better fashion, etc. But it had limitations.
First I didn't get it, then it sunk into me - once you have thousands of jobs - it's pointless to try and debug one of them (thousands, as in MapReduce thousands). Not that I've used the feature a lot, but tried it several times and worked fine.
It had nice integration with codesearch, where you can, given your "namespace" and "job" name, can set the codesearch to be "synced" to the CL it built your job + cherrypicks (I guess) - so the code you are looking is the code that your binary got compiled (another benefit of monolith depots, is that such things are easier there, or maybe it'll take a bit more work to get it working with multiple repos).
E.g. magic :)
Agreed. In general, every time you need to SSH anywhere, it means that there is some tooling you are missing.
In very rare cases, you do need to break the glass to deal with exceptional conditions, but this should be rare.
The in-built options (Basic auth, token auth, client cert) are not really scalable for a larger number of users, and the other options (webhook, OIDC) can be complex to setup.
So any relatively easy to use options for this would be welcome.
Here is an example of how to do this (OIDC). Notice the short and concise documentation on how to set this up. It isn't too bad at all if you're running your own apiserver already:
You can hook up an enterprise IdP like AD FS with that, without any additional software.
The second, perhaps less obvious, benefit is Teleport's built-in audit as shown in the video, i.e. all your interactions with all Kubernetes clusters are logged in one centralized place for audit purposes (and, in the case of the interactive sessions) can be replayed in a Youtube-like player.
But yes, if you don't need the audit, and you don't use SSH, and if you only have one k8s cluster, this may be an overkill.
K8s authentication methods that go inline with the apiserver, such as oidc or dex, achieve the same goals but are a bit of a different design flavor and arguably less clean from a threat or failure model perspective.
If your organisation has implemented an OIDC IdP, you can have the API server behind a Bastion like you suggest, and accessible through an authenticating reverse proxy (Beyondcorp style) WAF. You can even have token bound access/ID tokens which are backed by hardware TPMs if you're REALLY paranoid (upcoming spec). Plus because it's an open standard which K8s heavily embraces, you get rich group/user claims mapping in K8s allowing for logging and auditing.
Whenever there is a 'contact us' pricing page, I don't even bother. I can't even tell my boss to look into it if I don't know what the required budget will be, and I won't give out information to get contacted by sales teams and waste my time on a project that I wasn't even allocated time for.
Plus it won't even be only my time, it will be finance and maybe even legal. So now there are multiple people involved to get a quote before it's even figured out if the solution fits the budget or if it is even a fit for the problem (unless there is some sort of trial).
Whenever a project does this, it had better be the only thing capable of dousing a burning fire to get even considered.
EDIT: I understand that this is a common practice and a legitimate business model, so it's not aimed at Gravitational specifically. It just causes people to sometimes dismiss offerings that they would be legitimately interested in, otherwise.
For instance, Slack offers paid versions that are not yet at the Enterprise level(so less features), but you can at least get an idea, and also what is the pricing model (is it per seat? per server? per gb of storage? etc).