undefined | Better HN

0 pointslumost2y ago0 comments

An LLM is simply a model which given a sequence, predicts the rest of the sequence.

You can accurately describe any AGI or reasoning problem as an open domain sequence modeling problem. It is not an unreasonable hypothesis that brains evolved to solve a similar sequence modeling problem.

0 comments

timacles2y ago

> It is not an unreasonable hypothesis that brains evolved to solve a similar sequence modeling problem.

The real world is random, requires making decisions on incomplete information in situations that have never happened before. The real world is not a sequence of tokens.

Consciousness requires instincts in order to prioritize the endless streams of information. One thing people dont want to accept about any AI is that humans always have to tell it WHAT to think about. Our base reptilian brains are the core driver behind all behavior. AI cannot learn that

measured_step2y ago

How do our base reptilian brains reason? We don't know the specifics, but unless it's magic, then it's determined by some kind of logic. I doubt that logic is so unique that it can't eventually be reproduced in computers.

goatlover2y ago

Reptiles didn't use language tokens, that's for sure. We don't have reptilian brains anyway, it's just that part of our brain architecture evolved from a common ancestor. The stuff that might function somewhat similar to an LLM is most likely in the neocortex. But that's for neuroscientists to figure out, not computer scientists. Whatever the case is, it had to have evolved. LLMs are intelligently designed by us, so we should be a little cautious in making that analogy.

hutzlibu2y ago

"Consciousness requires instincts in order to prioritize the endless streams of information. "

What if "instinct" is also just (pretrained) model weight?

The human brain is very complex and far from understood and definitely does NOT work like a LLM. But it likely shares some core concepts. Neuronal networks were inspired by brain synapses after all.

timacles2y ago

> What if "instinct" is also just (pretrained) model weight?

Sure - then it will take the same amount of energy to train as our reptilian and higher brains took. That means trillions of real life experiences over millions of years.

jodrellblank2y ago

Not at all, it took life hundreds of millions of years to develop brains that could work with language, and took us tens of thousands of years to develop languages and writing and universal literacy. Now computers can print it, visually read it, speech-to-text transcribe it, write/create/generate it coherently, text-to-speech output it, translate between languages, rewrite in different styles, explain other writings, and that only took - well, roughly one human lifetime since computers became a thing.

stevenhuang2y ago

The real world is informational. If the world is truly random and devoid of information, you wouldn't exist.

goatlover2y ago

Information is a loaded word. Sure, you can say that based on our physical theories, you can think of the world that way, but information is what's meaningful to us amongst all the noise of the world. Meaningful for goals like survival and reproduction from our ancestors. Nervous systems evolved to help animals decide what's important to focus on. It's not a premade data set, the brain makes it meaningful in context of it's environment.

RandomLensman2y ago

In the broader sense that is tricky as accurate prediction is not always the right metric (otherwise we'd still be using epicycles for the planets).

lumostOP2y ago

It depends on the goal, epicycles don't tell you about the nature of heavenly bodies - but they do let you keep an accurate calendar for a reasonable definition of accurate. I'm not sure whether I need deep understanding of intelligence to gain economic benefit from AI.

j / k navigate · click thread line to collapse

0 comments

timacles2y ago

> It is not an unreasonable hypothesis that brains evolved to solve a similar sequence modeling problem.

The real world is random, requires making decisions on incomplete information in situations that have never happened before. The real world is not a sequence of tokens.

measured_step2y ago

goatlover2y ago

hutzlibu2y ago

"Consciousness requires instincts in order to prioritize the endless streams of information. "

What if "instinct" is also just (pretrained) model weight?

The human brain is very complex and far from understood and definitely does NOT work like a LLM. But it likely shares some core concepts. Neuronal networks were inspired by brain synapses after all.

timacles2y ago

> What if "instinct" is also just (pretrained) model weight?

Sure - then it will take the same amount of energy to train as our reptilian and higher brains took. That means trillions of real life experiences over millions of years.

jodrellblank2y ago

stevenhuang2y ago

The real world is informational. If the world is truly random and devoid of information, you wouldn't exist.

goatlover2y ago

RandomLensman2y ago

In the broader sense that is tricky as accurate prediction is not always the right metric (otherwise we'd still be using epicycles for the planets).

lumostOP2y ago

j / k navigate · click thread line to collapse