It's not like the other 90% of people lack the intelligence or whatever else to be much stronger than they are, but it requires extensive dedication, work, and suffering that many just uninterested in tolerating for the sake of improving at a single domain. I think your example largely proves the point. Anybody of average intelligence can obviously excel at undergraduate math if they're willing to dedicate themselves to it, but many people aren't. If somebody was failing at math it's probably because they were just treating it like you might e.g. literature, and trying to do cram sessions relatively shortly before each exam, whereas by the time somebody gets to stuff like diff eq math starts turning more into a puzzle game that requires developing things on a subconscious/intuitive level.
