One of the reasons people in power can behave so terribly towards specific groups is because they can't "just leave" like you suggest. You think the bully's going to choose the strongest person as a victim?
I try to pay the favor forward by speaking out and supporting folks who are treated badly by shitty leadership whenever I can.