if he was a white-supremacist who wrote a scientifically-backed manifesto about biological differences with his black coworkers, would you still be blaming the people who are 'shunning' him?
No, how they act is their fault.