The problem is that whatever group is considered evil can change day to day. Right now 99% of people, myself included, can agree that supremacists or neo nazis are bad people and output evil ideas. However, it wasn't too long ago that the majority of people were against gays getting married, or even different races getting married. When we set up precedent or infrastructure to harm a group outside of the rule of law just because they are assholes, we leave the opportunity open for those in power to remove other groups they don't agree with even when those other groups are not as clear cut evil