You need to replicate in Cloud too, most people tend not to because they think the cloud is magic, but it's computers and computers can fail- even if they're someone else's.

Also "if some component fails or the app is critical" has a lot of nuance, I agree with your sentiment but you should know:

1) Component failures in hardware are much rarer than you think

2) Component failures in hardware can be mitigated (dead ram, dead PSU, dead hard disk, even dead CPUs in some cases: all mitigated) The only true failure of a machine is an unmitigated failure due to not configuring memory mirroring or something' or a motherboard failure (which is extremely uncommon)

3) The next step after "single server" isn't "build a datacenter", it's buying a couple more servers and renting half a rack from your local datacenter, they'll have redundant power, redundant cooling and redundant networking. They'll even help you get set up if it's 2-3 machines with their own hardware techs.

I do this last one at a larger scale in Bahnhof.

also, $3000 will get you about 3-5 years out of hardware, at which point, yeah, you should think about upgrading, if for no other reason than it's going to be slower.