Would you be OK with me asking you to avoid using a standard U.S. English phrase so it's clearer for us Brits?
"Black cab driver" is part of our lexicon. It carries a lot of cultural meaning, and it's always phrased like that. You briefly misunderstood it as race-related, then you learned what it means. I don't see a reason to change anything.