History. There is an undeniable legacy of gender and race-based discrimination. This discrimination was absolutely laser-focused on gender and race, so is it fairer to simply ignore this and call it a wash?

For instance, some estimates of the value of free labor provided by slaves in the U.S. are upwards of $100 trillion, with an amplification effect (including an inverse one), compounding over generations.

>Do people really think that a white guy from some poor area has more privilege...

No. There are programs for people in rural areas, poorer areas, etc. To the extent that these are inadequate, they should also be addressed.

But, it's not just about economics. It's also about culture, socialization, etc.