If all you're using is VM's and their infinitely scaling ability, it can pretty easily become cheaper to just run your own hosts and manage a pool of not-yet-in-use hardware. Or to do something like setup greenfield stuff in AWS, then at the start of the next quarter, buy hardware for the stuff in AWS, migrate it to in-house, and start over again.
I don't find cloud-agnostic to be that compelling of a solution, generally. It really only makes sense to me at small footprints. Even then, I'd probably only suggest it if you needed a really small footprint (like 1-3 servers) across a large number of geographic sites, and for some reason you can't just use a CDN.
...eh. There's a fixed staffing cost of figuring out how to script out deployment to a new provider, and then you're pretty much done. Even if it's some podunk VPS host that doesn't provide an API or anything for automated setup, most of the effort of configuring and deploying to a server happens on the server itself over SSH anyway, so getting that server's (virtual or physical) hardware "powered on" is a pretty small fraction of that "staffing cost".
That is: if your sysadmin is worth one's salt, one will have very little trouble with this. And if you don't have a sysadmin (or someone with approximately that skillset)... well, that's gonna bite you for a lot more reasons than just being beholden to a specific cloud provider.
> which means not using value add services.
The term "value add services" is dependent on the notion that said services actually add value, particularly for a startup in your framed hypothetical. For example:
> No more SQS or the GCP/Azure equivalents, you have to run your own Kafka in case one of them goes evil.
Kafka arguably adds more value than any provider-specific "value add service" like SQS, assuming you actually need a full-blown message queue. Chances are, however, you don't need a full-blown message queue, and trying to architect your app around one - be it SQS, Kafka, or whatever else - is exactly the sort of distraction your fledgling startup can't afford.
> If all you're using is VM's and their infinitely scaling ability, it can pretty easily become cheaper to just run your own hosts and manage a pool of not-yet-in-use hardware.
I mean, if "cheap" is the only thing you care about then on-prem (or colocated) is pretty much the only way to go, period. Those "value add services" that cloud providers provide are themselves pretty expensive compared to just running your own version (even in a VPS, let alone on your own hardware).
But this doesn't address what happens when (not if) that datacenter has an outage, so you'll still end up needing at least two such sites if you need any sort of reliability. And maybe you don't need any sort of reliability - an early stage startup probably doesn't - but a business that already has customers and doesn't want to be on the hamster wheel of perpetually selling itself to investors just to "raise capital" and make ends meet probably should be a bit proactive about resiliency.
