The problem people have with Social Justice is that they are perceived as "going overboard"; they attach the same importance to staring at someone's chest that they would attach to grabbing said chest, which is equated to straight-up rape.
This is, of course, a vocal minority speaking--a group of people who attach their entire self-worth to the idea that they're fighting evil. They can't bear to think that the fight is over, because then their life becomes devoid of meaning, so they insist that the fight must be taken even further.
The issue is further complicated by what I call the Villain Effect. Because the other side is opposed to you, they are evil; therefore, any compromise is immoral. This happens on both sides, leading to a destructive stalemate in which everyone fights for an increasingly extreme version of their original vision, with no end in sight.