Its been a while since I read the literature, but if I remember correctly, this effect you are describing is usually controlled for. If I remember correctly, it is the inequality it self, that decreases quality of life, and is independent of the increase in wealth among the poorer public.

That is, over a certain baseline, an increase in wealth does not yield significant improvement in quality of life (i.e. diminishing returns). Redistributing the wealth to increase inequality will improve the collective quality of life, not because the general public has more money, but because there is more equality.

Again it is up to you if you don’t believe this, just be aware that your believes are contrary to the scientific consensus.