It’s all a little bit more agnostic than k8s but If you actually run consul + vault + nomad in production you know that managing it and repairing it can be a nightmare. GKE or EKS on the other hand feels like I almost shouldn’t even have a job.
In fact, two of the most high profile outages in the last two years(Robinhood March 2020 and whatever Roblox outage happened recently) we’re due to consul or vault or something going completely hare brained.
Nomad now has its own service discovery (along with secure variables coming in 1.4.0) so you won't actually need Consul and Vault anymore. This is perfect for smaller deployments like on a Raspberry Pi.
Also, I made some automation for setting up Nomad on a single Fedora CoreOS server (still kind of a work in progress) if anyone wants to give it a try.
Nomad, Consul and Vault work find at small to medium scales. Sometimes. If things are working really well and there aren’t any real problems on the network. It’s very fragile, though, and tends to shit itself under the slightest bit of abnormal pressure.
We migrated around 20 services within a couple of days, coming from a mixture of all sorts of AWS services and Hetzner infra.
Documentation was fairly good and the community was helpful.
(This is without consul or vault, we use native service discovery so far)
Obviously this could make a 180 easily because of how much control HashiCorp has, but, thus far, they have been pretty good stewards.
Setting up your own, non-managed Kubernetes cluster is definitely possible but I’d argue it’s on a whole different level of difficulty.