The problem is that that doesn't really work. The people working in the office don't get the benefit of in person social connection if others aren't coming in, too. Either you have people coming (win for the office people, loss for the remote) or you have people working from home (the reverse). Leaving it up to everyone to work it out on their own doesn't really solve the problem.
That being said, working it out "per team" is generally reasonable. I've had hybrid stints, and I always coordinated with my team so that we all came in on the same day. Then we could do the social thing together at the office.