Nomad + Consul is the happy path. Adding Vault into the mix is nice, but not required.

Consul by itself is the game-changer. Even in k8s it's a game-changer. It solves so many questions in an elegant way.

"How do I find and reach the things running in (orchestrator) with (unknown ip/random port) from (legacy)?" being the most important. You run 5 servers, and a relatively lightweight client on everything (which isn't even outright required, but it sure is useful!), and you get a _lot_ with that.

Consul provides multiple interfaces and ingress points to find everything. It also is super easy to operate, and has a pretty big community.

If you absolutely cannot have Consul, Nomad is still a really good batch job engine, and makes a very great "distributed cron," which is more extensible, scalable, and easy to use than something like Jenkins for the same task.

My team is pretty small (was 4 people, now 6) and we manage one of the worlds largest nomad and consul clusters (there are some truly staggeringly large users of Vault so I won't make that claim). Even when shit really hits the fan, everything is designed in a way that stuff mostly works; and there's enough operator friendly entry points that we can always figure out the problem.

Nomad Team Lead here.

Edit: just noticed an actual Nomad user replied as well, and I like their answer better. Consider mine an addendum. :)

Batch workloads rarely require Consul, but for deploying your standard network services on Nomad: Consul is basically required. You could likely use any number of service mesh systems instead (either as sidecars, Docker network plugins, or soon CNI), but you'll be doing a lot of research and development on your own I'm afraid.

The Nomad team is by no means opposed to becoming more flexible in the future (and indeed better CNI support is landing soon as a first step), but we wanted to focus on getting one platform right and a pleasure to use before trying to genericize and modularize it.

I'm sympathetic toward the idea of a system made of interchangeable parts, but I also kinda feel like it's a bit unrealistic, maybe? Even with well-defined interfaces, there will always be interop problems due to bugs or just people interpreting the interface specs differently. Every new piece to the puzzle adds another line (or several) to a testing matrix, and most projects just don't have the time and resources to do that kind of testing. It's unfortunate, but IMO understandable that there's often a well-tested happy path that everyone should use, even when theoretically things are modular and replaceable.