it's a pain because of NAT/port forwarding, not to mention the chore of making sure every device is up to date. You can get around the NAT/port forwarding issue by having a $5 VPS, but then you're essentially storing your password database on dropbox.
>Also, what is there to compromise on a machine that basically runs just sshd with password authentication disabled?
1. Same way that dropbox can be compromised: your account gets hacked or the provider gets hacked.
2. While I agree that a server with only sshd and auto-updates enabled would be pretty hard to compromise (foregoing the above), I doubt that's the typical setup. Most people probably have a "general purpose" VPS that they use to host all sorts of stuff, which means there's lots blindly typing in "npm install ..." or even "curl ... | sh" going on.