Taking more money from the poor makes them more poor.
Taking more from the rich 'harms' them less because of the margin utility of money: taking $1k from someone making $160k is qualitatively different from someone making $60k. Taking 20% from someone making $160k ($32k) is qualitatively different than taking 20% from someone making $60k ($12k).
So if you want to have X revenues for funding government, you can take it from those who have more marginal need of it, or those who have less.