The only such notices I've been truly annoyed by are the ones that say "now please wait 6-8 weeks for your unsubscribe request to be processed".
Otherwise, it's really a very small thing to deal with. And it can be nice to have that email in your archive for future reference, like if they start spamming you again.
Same here, doesn't bother me either. I think the people it bothers might be bothered partly because they were annoyed at being spammed in the first place, and this is just another drop in the bucket. Sure, it'll trigger annoyance, but the real cause lies elsewhere.
