undefined | Better HN

0 pointsCommieBobDole5h ago0 comments

This article suffers from two things:

First and most importantly, it's not really about LLMs, it's about AGI, and the second does not necessarily follow from the first; LLMs in their current state are pretty clearly not AGI, and most of the LLM-world progression in the last few years has been about better tooling/interfaces, refinements in training data and techniques and people learning how to use LLMs effectively rather than the huge leaps in fundamental capability that we saw in earlier years. It seems more likely that at this point, when AGI comes, it will be something entirely new or something that LLMs are only a component of, rather than "we built an LLM with ten trillion parameters and suddenly it became God".

Second, it's not even really about AGI, it's about AGI superintelligence. And more than that, it's about affordable AGI superintelligence, assuming that such a thing won't cost billions a year to operate.

0 comments

antonvs5h ago

> LLMs in their current state are pretty clearly not AGI

That depends a lot on definitions. It's artificial, it's very general, and by many measures it's intelligent - often superhumanly so, especially when compared to the average human.

That covers the A, the G, and the I. So why is it "clearly not AGI"?

avidiax4h ago

Present LLMs are quite good at interpolating, in fact, too good.

That's the source of hallucinations. A path can be found between A and B, even if A is the 12th century Chinese royal court and B is the Easter bunny.

Interpolation and rote knowledge are still very useful. Most cognitive tasks are like this.

The thing that LLMs are not presently good at is extrapolation. You can train an LLM on pre-1904 literature, but you won't get special relativity from it, at least not without a human to prompt it in just the right way.

You can have an LLM provide a "novel math proof", but you are necessarily discarding 100 or 1,000 "novel math mistakes". The process is more like a guided walk (like the A* algorithm), with human supervision and intervention, not an autonomous math genius.

"They" are, of course, working on it. But the present implementation has some severe structural limitations (such as an inability for new or discovered information to affect model weights) that make LLMs as a human replacement incomplete.

CharlesW5h ago

Ask your favorite SOTA LLM, "Why are SOTA LLMs very clearly not AGI?", and be enlightened.

j / k navigate · click thread line to collapse