I don’t see why children should have their opportunity limited by what their parents can or can’t do (or even refuse to do). How is that meritocracy or equality of opportunity? That’s just old fashion inheritance.
You are absolutely right. If playing some sports that only rich can afford makes you more likely to get into an ivy league school then may be hardships should too.