With Portainer and Swarm I've been forced to manually intervene with rollouts more times than I would have liked due to Swarm-specific errors and other quirky networking behaviour. At least it's simpler than Kubernetes!
Kubernetes solves a similar class of problems to Swarm, but in a much more complex way. Sometimes that complexity helps solve problems. For many organizations, though, Swarm would have been the better option.
Both Swarm and Kubernetes have their purposes. I'm sad that no option has popped up to replace Swarm in the "simple and easy container orchestration" space. Now it's either Kubernetes or... ECS, I guess.
I tried setting up nomad but couldn't figure out service discovery, external ingress / letsencrypt in the time I allotted myself...
I've had a hard time finding much via hn.algolia.com to back this up, other than some discussions in mid-2018, though.
Or are you talking about swarm the product (versus docker swarm mode)
https://pypi.org/project/docker-harbormaster
It's great if you want to run generic utilities at home (though I've used it at work in internal production and it was good), but it doesn't do ingress, so you have to bring your own.
It's basically a fancy/opinionated wrapper over "git pull && docker-compose up", with allowing you to specify all configuration in one file/repo.
Dokku does support both Kubernetes and Nomad as deployment targets, so it's not strictly single-server (though app builds currently are).
Does anyone have suggestions?
- Single-server and non-secure (no mTLS; no ACLs) clusters are super easy to set up and a great way to try out things before committing.
- ...However, enabling especially ACLs but also TLS on running clusters is going to be more hassle than simply setting up a fresh, properly bootstrapped cluster.
- Their minimum and recommended resource requirements are hugely inflated, you can generally get good mileage out of way less
- ...However: Do follow their advice on keeping nodes single-responsibility (ie don't run a consul server and nomad server on the same node; generally keep your servers dedicated and not running jobs)
- Consul and Vault integrations are generally rock solid.
- ...However: Nomad native service discovery is not yet (wrt consistency and template rewrites). Consul Connect may or may not have edge (it's some time since last we seriously tried it).
- learn.hashicorp.com has material for most things you want to do.
- Put all your ACL configuration in Terraform (or whatever else you have for the same purpose)
It's just that people need to start planning more before they dive into k8s, there's a limit to where and when k8s makes sense in your app. And most people could get away with the more traditional setup of Frontend, App and DB Backend for a service oriented architecture and still be able to scale up into k8s when necessary.
This is a pretty basic setup and with k3s one can be very productive at it. In fact, you will very likely hit quite a lot of k8s bottlenecks at Google scale if you throw everything at a single cluster.
I feel like we would actually be better off if people admit they'd rather manage all the complexity by hand all the tine, than spend a few hours reading the docs on k8s objects properties and lifecycles - that's really all there is to it for basic setups like k3s.
But for mature companies K8s can make a ton of sense.
