Yet ChatGPT totally - apparently - gets 1 + 1. In fact it aces the addition table way beyond what a child or even your average adult can handle. It's only when you get to numbers in the billions that it's weaknesses become apparent. One thing it starts messing up is carry-over operations, from what I can see. Btw. the treshold used to be significanly lower, yet that doesn't convince me in the least that it's made progress in its understanding of addition. It's still just as much in the fog. And it cannot introspect and tell me what it's doing so I can point out where it's going wrong.

But I think you are right in what you are saying. Basically it not 'seeing' math as a child does, is just another way to say that it doesn't undestand math. It doesn't have a intuitive understanding of numbers. It also can't really experiment. What would experimenting mean in this context? Just more training cycles. This being math, one could have it run random sums and give it the correct answer each time. That's one way to experiment, but that wouldn't solve the issue. At some point it would reach its capacity of absorbing statistical corelations to deal with numbers large enough. It would need more neurons to progress beyond that stage.

Btw. I found this relevant article: https://bdtechtalks.com/2022/06/27/large-language-models-log...