Nope. There are people (I'm one of them) who have sometimes extreme difficulty doing basic arithmetic manually and in the head. In my case it's not the full-blown dyscalculia[1] but to this day I haven't been able to learn full multiplication table for example, so forget about multiplying numbers in my head. I can do it on paper, but the whole process looks like a computer under heavy swapping. I even have hard time adding and subtracting numbers mentally. If I'm expected to produce a numerical result while being watched it all becomes a catastrophe because then I also get substantial anxiety for not being able to do what for most people is a simple thing. There's something about operations with concrete numbers (although I have no problem with numbers themselves) that turns my mind into mush.

OTOH, I have basically little problem with any other branch of mathematics. The more advanced the better, actually. In school it got easier as it got more advanced (although geometry was always easy no matter what level). Getting calculus in high school felt like I could suddenly breathe with full lungs. And I studied theoretical physics later at university. Apart from the basic reality that the math required is hard and needs a lot of work no matter how smart/gifted/whatever you are, I had no substantial problem with most of it. Abstract algebra, group theory, vector spaces and manifolds... oh the joys! Because none of it requires you to do any kind of mental arithmetical computation. This is not a unique experience, I once saw a TV report on a young successful astrophysicist with basically the same problem: Calculus on manifolds, GR field equations? Pfff... All day and every day. But, give her some numbers on a blackboard to multiply and she gets completely lost.

[1]: https://en.wikipedia.org/wiki/Dyscalculia