Not sure why you find that surprising. Being German is not written on your face. Since most Germans are white, most people will make the correct assumption that if someone is not white, there is a stronger likelihood that they are not German. The same happens in Japan with mixed race kids who get treated like foreigners even though they were born and spent their whole life in Japan. That's just how brains work.
If you had no prior assumption you could assume that nobody is who they seem to be and that would make things very complicated for everyday life.