Your comment is really quite out of touch of how layoffs actually affect people.
I’ve been through a few rounds now. Morale is essentially destroyed in the short term. Your team suddenly has a lot more work with no additional support or even acknowledgement that people are now slammed.
It’s not inconvenience. It’s a significantly negative change in the work environment, and a sign that maybe your company’s long-term prospects aren’t great. Of course good people leave in these situations.
Having a bunch of people leave due to RTO is different than having seemingly random colleagues laid off.
Both are not nice, RTO is more voluntarily and more avoidable since you can sometimes bargain for remote work to be a part of your contract and not just an oral agreement.
It's morally questionable to call for RTO only to get rid of people without technically having layoffs, but in countries like the US getting laid off is probably worse than having working conditions degrade to where you just find another job of your own volition since there's less urgency.
It is always just pure wishful thinking that "all the people you will loose when you alienate someone like me" are totally the best people out there.