What you’re describing is probably a mix of casteism + colourism.

There is no strict correlation between skin colour and caste. The current Prime Minister Narendra Modi is fair skinned and from a lower caste while a cricketer like Hardik Pandya who is darker skinned is from an upper caste.