I've worked remotely with a strong distributed team. It worked quite well, and I formed relationships on par with the year I worked in-person at another company.
That being said, I'm not very interested in it for my company since it just seems less fun to not be able to hang out with your employees in person. And fun is the reason I do this. I'm still on the fence about it though.
So if "fun" is the reason you do it, "fun" must also be the reason your employees do? Do you put this in your job ads as a required skill, doing it for fun, or is it perhaps something you don't say out loud?
Part of having "fun" is also feeling secure and safe in your job. At any rate, I know many people who find it more fulfilling to engage with their team locally, and I'm one of them, and I don't see what's wrong with that (especially since it is my company after all and I'm pretty sure I'm not breaking any laws by stating that).