Actually, vacation / time off shouldn't be an issue. I've been doing this for 15 years and now my product runs like clockwork. Generally unless I'm doing new dev work, I only have a few support emails per day. As long as I go on vacation somewhere where I can at least have wifi in the evenings and mobile data during the day to occasionally check my email, it's not a problem. Also I regularly take long breaks during the day, or take days off to go on trips somewhere. I think you need to do this otherwise you'll be in danger of burning out.
It also helps to plan for the worst. Right now my slight concern is lack of RAID on my server. However I back up every day to a separate disk on the server (plus offsite downloads every couple of weeks), so in the worst case it will probably just take me an hour or two to get most things up and running on a new server. Obviously not the best thing to happen on vacation though :)