So I ran a half-baked experiment here. Had a friend queue up a song (15 in total) in "HD" and not, play 30 seconds of one, pause for a few seconds, then play the same 30 seconds again, randomly choosing between the same quality (placebo-is) and the other.
Then I guessed "same or different" quality. Got about 85% right overall.
I'm sure it wasn't perfectly blind, but seems to indicate to me that it does in fact sound different. Now, I cannot say for sure it's not sound cards or other things in the output path causing differences as well.
Now if you asked me to pick which was HD and which was 320, I did worse than random. So I am not claiming that HD sounds "better", just different.