I’ve always liked the phrase “money can’t buy happiness but it can make you less miserable.” I think the main idea is that there are certain circumstances that can bias you into an unhappy state, but merely removing what makes you miserable isn’t enough to make you happy.

Happy or less annoyed? I find when I was truly happy I cared less about external influences. For example, if it's hot, a person is annyoing, someone almost hits me with a car, or it my manager wants me to do something boring.

So maybe what some people think are the sources of their unhappiness is actually just making their unhappiness more obvious.