While it's hardly fair I have to agree. I'm acquainted with a few and they are generally cool people but part of the brony identity does seem to be tied directly to the stigma against them. I wonder if feeling like they have to hide it makes them more resistant to hiding it?
It's such a silly thing, but humans are animals and we have annoying or shitty things like pack/tribe mentalities.