The difference is where you start. If you start lower middle class or working class, odds are working hard will get you at best stable employment and the ability for your children to do something. Start upper middle class or upper class and you're going to the best high schools, best colleges, and have a shot at politics or being worth 70 billion dollars.
I've met too many poor and undereducated people who are as capable as my coworkers who went to Harvard, but the poor and uneducated have to work extremely hard, be extremely lucky, and do things others can't imagine to succeed.
The system is rigged against them, and it's hard for the well-off to admit that they got a 30-yard head start or for them to begin to understand it. Maybe that's why the system remains as it is, with the same millionaires and billionaires telling people who make 30k a year how they should live.