It has more to do with the psychology of the person who talks about others that "don't vote in their self-interest". That person, invariably, thinks of others as robots that should do what he wants them to do, because of course what he wants is best for everyone. He cannot imagine that people external to himself have any real interests at all. Everyone in the world must, as some precondition of the universe, be interested in all the same things and in all the same ways as he himself does.

So when someone "votes against their self-interest", this person tends to think of those others as malfunctioning. Perhaps they're too stupid to correctly deduce the path to achieving the results they want. Though he might be willing to consider they're mentally ill.

If he were forced (somehow) to consider that other people want things different from what he wants, it could be some sort of existential crisis as far as he's concerned. How could two competing interests even exist in a sane or fair universe, and which should prevail if they are mutually exclusive? What if, somehow, his own interests were destined to lose out?