I have experience with Consul, Vault and Nomad.
Due to security and performance reasons, we need to have most of our services hosted on premises.
I've been using a on-premise Nomad deployment in a bunch of greenfield projects and so far I'm very satisfied on how well it integrates with Consul/Fabio/Vault and how simple is to operate the cluster.
We are not using any fancy feature and we are basically using Nomad to decouple the Software from the hardware where the Software runs. The deployment times are faster than using ansible and we can leave the scheduler to decide where the software should be deployed. Nomad also supports scheduling of raw commands.
Lastly, Nomad, when integrated with Consul/Fabio works in a multi-datacenter/multi-site topology out of box. This is specially important for us.
However due to the buzz around Kubernetes I'm afraid of making the wrong choice in the long term and invest time in something that will eventually die or become irrelevant. There is also much more information available about Kubernetes than Nomad.
On the other hand, I fell that Kubernetes is a bit more complex to operate/maintain than Nomad. I might be wrong.
My questions would be:
- If you have a on-premises Kubernetes deployment, is there a lot of overhead in operating it? Can you tell me a little bit about some specific situations where the cluster required manual intervention?
- Do you use Kubernetes in a multi-data center/multi-site configuration? Any thoughts on that?
- Do you use Nomad, used in the past? What works/worked and what doesn't/didn't?
