undefined | Better HN

0 pointsanon3738391y ago0 comments

Perhaps the system should be designed to equivocate on any conclusions, while prioritizing display of the source material. “Source X appears to state a rule requiring 2% shareholders to report abc, but I can’t say whether it applies: [Block quote Source X].”

0 comments

Terr_1y ago

That would be nice, but I cynically suspect it's not something LLMs are constitutionally able to provide.

Since they don't actually model facts or contradictions, adding prompt-text like "provide alternatives" is in effect more like "add weight to future tokens and words that correlate to what happened in documents where someone was asked to provide alternatives."

So the linguistic forms of cautious equivocation are easy to evoke, but reliably getting the logical content might be impossible.

anon373839OP1y ago

I agree, it is unlikely we’ll be able to get LLMs to provide “informed uncertainty” because they can’t interrogate any internal confidence in the correctness of the output.

But I wonder if tuning the output to avoid definitive statements would be beneficial from a UX perspective.

Terr_1y ago

I think it would help curb people over-trusting the model, yeah.

Heck, imagine how terrible the opposite would be: "When answering, be totally confident and assertive about your conclusions."

j / k navigate · click thread line to collapse

0 comments

Terr_1y ago

That would be nice, but I cynically suspect it's not something LLMs are constitutionally able to provide.

So the linguistic forms of cautious equivocation are easy to evoke, but reliably getting the logical content might be impossible.

anon373839OP1y ago

I agree, it is unlikely we’ll be able to get LLMs to provide “informed uncertainty” because they can’t interrogate any internal confidence in the correctness of the output.

But I wonder if tuning the output to avoid definitive statements would be beneficial from a UX perspective.

Terr_1y ago

I think it would help curb people over-trusting the model, yeah.

Heck, imagine how terrible the opposite would be: "When answering, be totally confident and assertive about your conclusions."

j / k navigate · click thread line to collapse