I did a blind test when I was in my 20s, and while on a couple of tracks I could actually tell the difference between 320kbps and the original, I did have to concentrate. And I couldn't really have said that one was necessarily better than the other; the effect was as if one type of noise-y sound was being replaced with a subtly different type of sound with the same noise-y quality. Different, but overall the same.

Listening to the diff of one of those tracks today was interesting! All I can hear is the drums... and where the sound I'm thinking of plays, it sounds like rather quiet interference! But the drums as I recall sounded absolutely identical. Interesting that the ears can detect one thing but not the other.

(I didn't bother to re-run the full comparison, as I'm no longer in my 20s. One good (?) thing about getting old is that your hearing deteriorates, and issues such as this become moot. You can also afford the disk space to just compress everything at 320kbps. Then you don't have to worry about it, and it fits OK on your phone too.)