Both responses address the problem with a human making a biased decision based on race, which I think mostly we all agree would be bad.
The question I was posing is different, though, because this was discussing an AI system that looked at the underlying [in this case, recidivism] data which had race and race-adjacent information removed, and the AI has effectively rediscovered the concept of "race" by connecting it to some set of attributes of the actual [in this case, recidivism-predicting] features. If the AI were to determine such a link, that doesn't make its results biased, it just makes them uncomfortable. It's not clear to me that in such a case that would mean that we should remove those [recidivism-predicting] features from the dataset just because they ended up being correlated to race.