Because you're white, he assumes you agree with him. In a lot of the US, 'thank god you're not dating a black girl' will only cause discomfort if they think they're in mixed company, not because they think it's wrong. Once the person who they don't know about leaves, so does any discomfort about saying what they really mean.
He doesn't say that shit to me anymore not because it's wrong, but because he's figured out I don't agree with him and look at him like the idiot he is.