For much of human history that has been a big filter, and it still is in parts of the world today. If you send your child to an Ivy instead of a state school, part of what you are paying for is your child fraternizing with a more affluent and well-connected group for 4 years. We certainly don't have any evidence that the Ivies teach better than other schools, and I have seen little demand for such evidence.
I'm not sure why you're explaining the allure of prestigious schools to me. Wealth correlates with success. Yet, I don't go on first dates and ask about family wealth. Can you guess why?