That doesn't work for multiple people in the same room all wearing AirPods. Everyone's mic picks up everyone's voice, not just the "real" speaker.
And a lot of meetings have most (e.g. 10) people in a single room, with another handful (e.g. 5) of remote participants.