Exactly, much like a chess bot can play perfectly without what humans would call thinking.
I think (ironically) we'll soon realize that there is no actual task that would require thinking as we know it.
> The question of whether computers can think is like the question of whether submarines can swim
It has only become more relevant.
If that were true, there would be no point in studying or doing any LSAT preparation. Writing practice exams would be of no benefit.
As others have said elsewhere, the issue remains accuracy. I wish every response comes with an accurate estimation of how true the answer is, because at the moment it gives wrong answers as confidently as right ones.
I can remember my GRE coach telling me that it was better to confidently choose an answer I only had 50% confidence in, rather than punt on the entire question.
AIs hallucinate because, statistically, it is 'rewarding' for them to do so. (In RLHF)
Obviously not, since GPT-4 doesn't have general intelligence. Likewise "common sense," "knowledge about the world," nor "reasoning ability."
As just one example, reasoning ability: GPT-4 failed at this problem I just came up with: "If Sarah was twice as old as Jimmy when Jimmy was 1/3 as old as Jane, and Jane is as much older than Sarah as Sarah is older than Jimmy, and Sarah is now 40, how old are Jane and Jimmy?"
First, every answer GPT-4 came up with contradicted the facts given: they were just wrong. But beyond that, it didn't recognize that there are many solutions to the problem. And later when I gave it an additional constraint to narrow it to one solution, it got the wrong answer again. And when I say "wrong," I mean that its answer clearly contradicted the facts given.
