The point is that brain is an organ like any others, and thus also affected by the evolutionary pressure. Nothing is truly equal at birth between humans (or animals) except for twins so it’s really specious to consider that intelligence, which is based on a physical organ, is somehow magically exactly the same for everyone. And once the fact that there is indeed individual variations is accepted, it’s not too far stretched to understand why there could be a different average intelligence between population.

The second thing is most people interpret mean (average) wrongly. It does not mean that every individual of group X is less intelligent than group Y. If all those activists had a better mathematical education they could probably deal better with the facts instead of harassing people who state them.