undefined | Better HN

0 pointsberniedurfee2y ago0 comments

Lol, yes, in fact, I was reacting to the article.

The point I was trying to make is that I think better LLMs won’t lead to AGI. The article focused on the mechanics and technology, but I feel that’s missing the point.

The point being, AGI is not going to be a direct outcome of LLM development, regardless of the efficiency or volume of data.

0 comments

ctoth2y ago

I can interpret this in a couple different ways, and I want to make sure I am engaging with what you said, and not with what I thought you said.

> I think better LLMs won’t lead to AGI.

Does this mean you believe that the Transformer architecture won't be an eventual part of AGI? (possibly true, though I wouldn't bet on it)

Does this mean that you see no path for GPT-4 to become an AGI if we just leave it alone sitting on its server? I could certainly agree with that.

Does this mean that something like large language models will not be used for their ability to model the world, or plan, or even just complete patterns as does our own System one in an eventual AGI architecture? I would have a lot more trouble agreeing with that.

In general, it seems like these sequence modelers that actually work right is a big primitive we didn't have in 2016 and they certainly seem to me as an important step. Something that will carry us far past human-level, whatever that means for textual tasks.

To bring it back to the article, probably pure scale isn't quite the secret sauce, but it's a good 80-90% and the rest will come from the increased interest, the shear number of human-level intelligences now working on this problem.

Too bad we haven't scaled safety nearly as fast though!

berniedurfeeOP2y ago

Yes, I suppose my assertion is that LLMs may be a step toward our understanding of what is required to create AGI. But, the technology (the algorithms) will not be part of the eventual solution.

Having said that, I do agree that LLMs will be transformative technology. As important perhaps as the transistor or the wheel.

I think LLMs will accelerate our ability as a species to solve problems even more than the calculator, computer or internet has.

I think the boost in human capability provided by LLMs will help us more rapidly discover the true nature of reasoning, intelligence and consciousness.

But, like the wheel, transistor, calculator, computer and internet; I feel strongly that LLMs will prove to be just another tool and not a foundational technology for AGI.

j / k navigate · click thread line to collapse