undefined | Better HN

0 pointsjltsiren3y ago0 comments

I think internal understanding requires internal processing.

According to this functional definition, the way we are currently using language models basically excludes understanding. We are asking them to dream up or brainstorm things – to tell us the first things they associate with the prompt.

Maybe it's possible to set up the system with some kind of self-feedback loop, where it continues evaluating and improving its answers without further prompts. If that works, it would be one step closer to a true AGI that can be said to understand things.

There is a lot of confusion around the Chinese Room Argument. I think it makes a valid point by demonstrating that input/output behavior alone is insufficient for evaluating whether a system is intelligent and understands things. In order to do that, we need to see (or assume) the internal mechanism.

0 comments

chpatrick3y ago

> Maybe it's possible to set up the system with some kind of self-feedback loop, where it continues evaluating and improving its answers without further prompts.

It can do that while it generates output. Humans do the same thing when they figure out what they really mean while they're trying to express it.

jltsirenOP3y ago

I was thinking more about the equivalent of a human noticing that the initial answer they were going to give is wrong and then thinking about the topic for a while before coming up with a better answer.