undefined | Better HN

0 pointsbeering4y ago0 comments

> In my mind, understanding a thing means you can justify an answer.

What if the language model can generate a step-by-step explanation in the form of text? [0]

There's no guarantee that the reasoning was used to come up with the answer in the first place, and no proof that the reasoning isn't just the product of "a really fancy markov chain generator", but would you accept it?

We're really walking into Searle's Chinese Room at this point.

[0] https://nitter.hu/kleptid/status/1284069270603866113#m

0 comments

naresh_xai4y ago

Umm, no there are clear verification methods for Explainable AI techniques today. One way to check the justification would be if things which were important in the justification were removed in some sense, then would the output change signficantly. Sort of like a sensitivity test for justification.

6gvONxR4sf7o4y ago

Searle's Chinese Room is exactly why I started thinking of understanding this way. It convinced me that a one-size-fits-all notion of understanding isn't useful. But it also made me think that understanding "with respect to system X" is useful.

If you can challenge an answer and get justification expressed in the form of X, then it understands with respect to X. A step-by-step text explanation is one form of X.

> ... but would you accept it?

This is all to sidestep questions of whether you accept X as "real" understanding or not. :D

piyh4y ago

>There's no guarantee that the reasoning was used to come up with the answer in the first place, and no proof that the reasoning isn't just the product of....

You're holding machines to a higher standard than we hold people.

j / k navigate · click thread line to collapse